an.build.shot_cache#

The content-keyed shot cache: ADR 0004’s first slice, behind the incremental= seam.

render_project asks an incremental engine, per shot, for a plan: the shot’s key, and — when an entry with that key exists — the rendered shot to reuse. It renders only the misses, and hands each fresh render back to be recorded. The engine is a seam (incremental= on an.render.render), and ShotCache is its built-in default; an nw-backed engine is the planned alternative (ADR 0004 decision 5).

**Data model: lacing’s** (decision 4). An entry is a ShotArtifact — a lacing.Artifact (asset_id = sha256 of the mp4, W3C-PROV provenance whose was_derived_from lists the key’s input digests) plus the named parts of its key and the timings that produced it. Entries live in a lacing.ArtifactStore: a catalog (key -> record) and a content-addressed blob store, both injected dol mappings. In a project the store is mall["shot_cache"] (artifacts/shot_cache/{catalog,blobs}/).

An ASSEMBLED film (transitions, a sound layer) is a stream-copy concat of segments (an.assemble, an#260): a shot no transition touches needs only its mp4, and a shot one touches also needs its parts — its body, encoded once, and the PNGs inside its transition window — a second entry, <key>.parts.<code>.h<head>.t<tail>, of a few MB. So such a film reuses its shots by default. (The older whole-frames entry, <key>.frames, is read only by a caller that asks for it with needs_frames=True; the render loop no longer does.)

Each cached render also records a root (root.<digest>): which entries a render of output <name> under one set of render knobs on one machine used. Roots are what an.build.gc keeps alive, beside what the current scene reaches.

Invalidation is by digest, never by deletion (decision 6): a changed input is a different key, and the old entry simply stops being asked for. The pre-cache artifacts/shots/<shot.id>.mp4 archive is not read. Collecting unreachable entries is a separate, explicit command: an cache gc (an.build.gc).

>>> from an.build.shot_cache import BuildReport, ShotOutcome
>>> r = BuildReport([ShotOutcome("a", "cutout", "reused", key="k" * 64),
...                  ShotOutcome("b", "cutout", "rendered", key="j" * 64, reason="new or changed")])
>>> r.summary()
'2 shot(s): 1 rendered (b), 1 reused (a); not reused: new or changed (b)'

Module Attributes

`SHOT_CACHE_STORE`	The mall key of the shot cache.
`PARTS_INFIX`	what an assembled film takes from a shot its transitions touch — its body, encoded once, and the PNGs inside its window.
`ROOT_PREFIX`	what one render used (see `ShotCache.record_root()`).

Functions

`default_environment_digest`(renderer_name)	The digest of `renderer_name`'s registered environment probe, once per process.
`explain_change`(before, before_reads, after, ...)	Why a shot keyed `after` is not the entry keyed `before`, in words.
`human_bytes`(n)	`n` bytes for a person: 1024-based, one decimal.
`in_memory_shot_cache_store`()	A shot cache held in dicts — for tests, and for a mall with no disk.
`machine_id`()	A short digest naming this machine (its host name and hardware address), so each machine's renders of a synced project keep a root of their own.
`project_id`(project_root)	A short digest naming a project by its resolved directory, so two projects sharing one cache store keep a root each (`""` for none).
`parts_entry_id`(key, window)	The catalog id of the parts entry of shot `key` for `window`.
`resolve_incremental`(incremental)	`incremental=` → an engine, or `None` for "render every shot cold".
`shot_artifact_type`()	The record type (`lacing.Artifact` subclass), built on first use.
`shot_cache_store`(root)	A filesystem shot cache under `root`: `catalog/` + `blobs/` (lacing's layout).
`store_usage`(store)	`(bytes, entries)` of a shot cache store: every blob byte on disk (or, off disk, every distinct blob its records name) and its catalog entries.

Classes

`BuildReport`([outcomes, store_bytes, ...])	Every shot's outcome, in timeline order.
`IncrementalEngine`(args, *kwargs)	The `incremental=` seam of `an.render.render` (ADR 0004 decision 5).
`ShotCache`([store, environment, ...])	The built-in engine: look a shot's key up in an ArtifactStore; record misses.
`ShotOutcome`(shot_id, renderer, status[, ...])	What happened to one shot in one render, with its wall times (seconds).
`ShotPlan`(shot_id, renderer, key[, inputs, ...])	The engine's answer for one shot: its key, and what to reuse if anything.

Exceptions

ShotCacheWarning

The shot cache could not do something it should have; the render still stands.

class an.build.shot_cache.BuildReport(outcomes=<factory>, store_bytes=None, store_entries=None)[source]#

Bases: object

Every shot’s outcome, in timeline order.

store_bytes: int | None = None#: The shot cache’s size after the render, when the engine measured it.

store_line()[source]#

How big the shot cache is after the render, or "" if unmeasured.

Return type:: str

>>> BuildReport(store_bytes=3 * 1024**2, store_entries=4).store_line()
'3.0 MB in 4 entries'

summary()[source]#

One line: what was rendered, what reused, and WHY each rendered shot was not reused — a cache that silently re-renders everything reads as a broken cache (an#243 review, R2-1).

Return type:: str

>>> BuildReport([ShotOutcome("a", "cutout", "uncached", reason=FRAMES_NOT_CACHED)]).summary()
'1 shot(s): 1 rendered (a), 0 reused (-); not reused: frames not cached: film has transitions/sound; pass --cache-frames (a)'

timing_table()[source]#

A Markdown table of the per-shot wall times.

Return type:: str

class an.build.shot_cache.IncrementalEngine(*args, **kwargs)[source]#

Bases: Protocol

The incremental= seam of an.render.render (ADR 0004 decision 5).

begin once per render with the project mall (and its root, for the root files every shot depends on); plan once per shot, BEFORE any shot renders (in the calling thread); record once per shot that was rendered (possibly from a worker thread); finish returns the report.

Two hooks are OPTIONAL (the render loop calls them when an engine has them): record_parts(plan, parts) once per rendered shot whose film window is not whole (an#260), and record_root(output_name, profile=..., output=...) once the film is delivered (what the garbage collector keeps).

an.build.shot_cache.PARTS_INFIX: str = '.parts.'#

what an assembled film takes from a shot its transitions touch — its body, encoded once, and the PNGs inside its window. See parts_entry_id().

Type:: The catalog id infix of a shot’s PARTS entry (an#260)

an.build.shot_cache.ROOT_PREFIX: str = 'root.'#

what one render used (see ShotCache.record_root()).

Type:: The catalog id prefix of a ROOT

an.build.shot_cache.SHOT_CACHE_STORE: str = 'shot_cache'#: The mall key of the shot cache.

class an.build.shot_cache.ShotCache(store=None, *, environment=<function default_environment_digest>, dependencies=<function project_dependencies>, fallback=<function every_asset_digest>, record_reads=True, cache_frames=False)[source]#

Bases: object

The built-in engine: look a shot’s key up in an ArtifactStore; record misses.

store is the injected lacing.ArtifactStore; None means the mall’s shot_cache (resolved in begin()), and a mall without one renders every shot. environment(renderer_name) -> digest is the environment seam — injectable so a test (or a remote-render backend) can state its machine rather than probe this one. Three dependency seams (see Dependencies):

record_reads (on by default) keys a shot whose keyer is registered with records_reads=True on the asset entries its compile read (an.build.reads, an#316: the assets part), so an edit to an asset re-renders only the shots that read it;
fallback is the assets part of every OTHER shot — a keyer that cannot vouch for its reads, or every shot under record_reads=False: by default every asset in the project (decision 3’s first slice);
dependencies is what EVERY shot depends on (the project part): by default the library lockfile. None drops it, and fallback=None keys an unrecorded shot on its own parts alone — use either knowingly.

An engine built with other seams writes keys an cache gc (which recomputes the current scene’s keys with the defaults) does not reach: collect such a cache with the same engine. cache_frames also stores each shot’s whole PNG sequence when a caller plans with needs_frames=True. The render loop no longer does (an#260): an assembled film takes a shot’s mp4 and, at a transition, its parts (window=), which are cached by default at a few MB. Kept for callers of the engine that need every frame; an cache gc collects old frames.

After a render, report holds what happened to each shot.

entry_ids(shot, renderer, ctx, *, window=None)[source]#

The catalog ids a plan() of this shot would read — computed by the same code, looking nothing up and rendering nothing. What an.build.gc keeps for the current scene. Call begin() first.

Return type:: list[str]

plan(shot, renderer, ctx, *, needs_frames=False, window=None, force=False)[source]#

window is what an assembled film needs from this shot (an.assemble.ShotWindow; None for a plain concat): a shot whose window is not whole is reused only when its parts entry is there too.

Return type:: ShotPlan

record_parts(plan, parts)[source]#

Store a freshly rendered shot’s an.assemble.ShotParts under plan.parts_id: its body mp4 and its window’s PNGs, one stored zip. A failed write never fails the render (the next render renders it).

Return type:: None

record_root(output_name, *, profile, output)[source]#

Record what this render used, as a ROOT for an.build.gc; return its id.

A root is keyed by the project (project_id()), the output name, the render profile (the knobs as passed — None meaning “the scene’s own”) and this machine (machine_id()), so the next render of the same output with the same knobs on the same machine replaces it — a browser upgrade included — while a render on another machine of a synced project, or of another project sharing the store, keeps its own. Its record is a lacing.Artifact of the delivered film, derived from the shot keys. Also measures the store, for the summary.

Return type:: str | None

exception an.build.shot_cache.ShotCacheWarning[source]#

Bases: UserWarning

The shot cache could not do something it should have; the render still stands.

class an.build.shot_cache.ShotOutcome(shot_id, renderer, status, key=None, reason='', key_s=None, compile_s=None, render_s=None, cached_render_s=None)[source]#

Bases: object

What happened to one shot in one render, with its wall times (seconds).

key_s is the whole key computation, of which compile_s is the compile; render_s is this render’s wall time (None when reused) and cached_render_s the wall time of the render being reused.

class an.build.shot_cache.ShotPlan(shot_id, renderer, key, inputs=<factory>, reads=<factory>, code=<factory>, cached=None, reason='', key_s=None, compile_s=None, cached_render_s=None, needs_frames=False, window=None, parts_id=None, parts=None)[source]#

Bases: object

The engine’s answer for one shot: its key, and what to reuse if anything.

code: dict[str, str]#: The render path’s modules, digested ({module: digest}, an#395).

parts: Any = None#: The reused parts (an.assemble.ShotParts), materialised for this plan.

parts_id: str | None = None#: The catalog id of this shot’s parts entry, when its window is not whole.

reads: dict[str, str]#: The asset entries the shot read, digested ({"store/key": digest}).

window: Any = None#: What an assembled film needs from this shot (an.assemble.ShotWindow); None for a film that is a plain concat of shot mp4s.

an.build.shot_cache.default_environment_digest(renderer_name)[source]#

The digest of renderer_name’s registered environment probe, once per process.

Memoised per renderer NAME (each backend has its own machine: Chromium and ffmpeg for cut-out, a Manim install for Manim), and once per process, because the cut-out probe launches a browser and encodes a frame. So a long-lived host does not see a playwright install or a brew upgrade made after its first render: restart it, or pass force_render. A renderer with no probe has the empty environment.

Return type:: str

an.build.shot_cache.explain_change(before, before_reads, after, after_reads, *, before_code=None, after_code=None)[source]#

Why a shot keyed after is not the entry keyed before, in words.

The assets it read that moved come first, by name (asset changed: props/logo), then every other key part that moved, appeared or went, by PART_LABELS (an optional part — fonts, runtime_extensions — comes and goes with what the shot draws). The parts that follow from an asset (FOLLOWS_ASSETS) are left out when one is named. Both keys are of one composition: a different key_version is KEY_FORMAT_CHANGED, decided before this is called.

Return type:: str

>>> explain_change({"compiled": "a", "assets": "x"}, {"props/logo": "1"},
...                {"compiled": "b", "assets": "y"}, {"props/logo": "2"})
'asset changed: props/logo'
>>> explain_change({"knobs": "a", "compiled": "c"}, {}, {"knobs": "b", "compiled": "c"}, {})
'render settings changed'
>>> explain_change({"compiled": "a"}, {}, {"compiled": "a", "fonts": "f"}, {})
'the system fonts changed'
>>> explain_change({"assets": "a"}, {}, {"assets": "b"}, {})  # an unrecorded shot
'a project asset changed'
>>> explain_change({"code": "a"}, {}, {"code": "b"}, {},
...                before_code={"an.motion": "1"}, after_code={"an.motion": "2"})
"an's render code (an.motion) changed"

an.build.shot_cache.human_bytes(n)[source]#

n bytes for a person: 1024-based, one decimal.

Return type:: str

>>> human_bytes(0), human_bytes(1536), human_bytes(5 * 1024**3)
('0 B', '1.5 KB', '5.0 GB')

an.build.shot_cache.in_memory_shot_cache_store()[source]#

A shot cache held in dicts — for tests, and for a mall with no disk.

Return type:: Any

an.build.shot_cache.machine_id()[source]#

A short digest naming this machine (its host name and hardware address), so each machine’s renders of a synced project keep a root of their own. A digest, never the names themselves.

Return type:: str

>>> len(machine_id())
16

an.build.shot_cache.parts_entry_id(key, window)[source]#

The catalog id of the parts entry of shot key for window.

The id names the window (the same shot cut for another neighbour is a different entry) and the digest of the code that cuts and encodes the parts (parts_code_digest()): a parts entry is produced by an.assemble, not by the renderer, so the renderer’s own code key part does not cover it.

Return type:: str

>>> from an.assemble import ShotWindow
>>> parts_entry_id("k" * 64, ShotWindow(frames=30, head=0, tail=4)).endswith(".h0.t4")
True

an.build.shot_cache.project_id(project_root)[source]#

A short digest naming a project by its resolved directory, so two projects sharing one cache store keep a root each ("" for none).

Return type:: str

>>> project_id(None)
''
>>> len(project_id("."))
16

an.build.shot_cache.resolve_incremental(incremental)[source]#

incremental= → an engine, or None for “render every shot cold”.

True is a fresh ShotCache over the mall’s store; False or None is off; anything else must be an IncrementalEngine.

Return type:: IncrementalEngine | None

>>> resolve_incremental(False) is None
True
>>> isinstance(resolve_incremental(True), ShotCache)
True

an.build.shot_cache.shot_artifact_type()[source]#: The record type (lacing.Artifact subclass), built on first use.

an.build.shot_cache.shot_cache_store(root)[source]#

A filesystem shot cache under root: catalog/ + blobs/ (lacing’s layout).

Return type:: Any

an.build.shot_cache.store_usage(store)[source]#

(bytes, entries) of a shot cache store: every blob byte on disk (or, off disk, every distinct blob its records name) and its catalog entries.

On a filesystem store this is a directory listing, not a read of every record, so the render summary can afford it on every render.

Return type:: tuple[int, int]