lacing#
lacing — interval annotation system.
Standoff, interval-keyed annotations with rational time, ELAN tier
stereotypes, Allen’s interval algebra, and a MutableMapping facade.
Quick start:
>>> from lacing import RationalTime, TimeInterval, Annotation, MemoryStore
>>> # Load a TextGrid, query overlaps, save as WebVTT — see misc/docs/.
Read CLAUDE.md and misc/docs/Lacing Development Roadmap.md for the
full story. .claude/skills/ contains the rules.
Functions
|
Return the canonical |
|
Return the canonical |
|
Return the SHA-256 hex digest of |
|
Return the SHA-256 hex digest of |
|
Register a forward store migration from |
|
Migrate a |
|
Rebuild a store by replaying |
|
Register a processor under |
Names of every registered processor, sorted. |
|
|
Run a processor in the current event loop. |
|
Run a processor synchronously and return its result. |
|
Return a tracer or a no-op fallback. |
|
Open a span on |
|
Decorator: wrap a function in a span on |
|
Add OpenTelemetry instrumentation to a FastAPI app. |
Quick check: is OTel installed AND a TracerProvider configured? |
|
|
Cohen's kappa for two annotators on a categorical label. |
|
Krippendorff's α across any number of annotators. |
|
Intersection-over-Union for two time intervals. |
|
Mean IoU between two sets of intervals via greedy best-match. |
|
Register |
|
Register a forward migration from |
|
Return the JSON Schema for the body model registered at |
|
Write every registered schema as JSON files under |
|
Render an annotation graph as a human-readable artifact exhibit. |
|
Migrate |
|
Validate |
Classes
|
A point in time as |
|
A half-open interval |
|
A named annotation layer with optional parent and stereotype. |
|
Constraints on how a child tier relates to its parent tier. |
|
The single annotation envelope. |
|
Reference to a region of a content-addressed media asset. |
|
Reference to a node in a structured scene/document graph. |
|
Reference to another annotation (for discussion threads, review, derivations). |
|
W3C PROV-O subset, embedded inline on every annotation. |
|
A content-addressed generated file with provenance. |
|
Who owns the bytes of an acquired artifact, and on what terms. |
|
Facade over an artifact |
|
The thirteen Allen relations. |
|
Protocol for any interval-keyed annotation store. |
|
|
|
An |
|
SQLite-backed |
|
Append-only log of mutations. |
|
One row of the op-log. |
Simple list-backed op-log. |
|
|
Op-log backed by a SQLite table. |
Exceptions
Raised when a rate or seconds conversion would lose precision. |
|
An annotation |
|
Raised when opening a |
|
Raised when a store migration step is missing or fails. |
|
Raised when a registered processor's invocation fails. |
|
Raised when a body fails validation against its registered schema. |
|
Raised when an export would write an empty registry over real artifacts. |
|
Raised when an annotation's body_schema_uri has no registered model. |
|
Raised when a migration step is missing or fails. |
- class lacing.AllenRelation(*values)[source]#
-
The thirteen Allen relations.
Symbols match Allen (1983); inverse pairs end in
i.
- class lacing.Annotation(**data)[source]#
Bases:
SchemaModelThe single annotation envelope.
bodyis typed bybody_schema_uri.- property interval: TimeInterval | None#
the reference’s interval, if any.
- Type:
Convenience
- class lacing.AnnotationRef(**data)[source]#
Bases:
SchemaModelReference to another annotation (for discussion threads, review, derivations).
- class lacing.Artifact(**data)[source]#
Bases:
BaseModelA content-addressed generated file with provenance.
asset_idis the SHA-256 hex digest of the artifact’s bytes. Two artifacts with the sameasset_idare byte-identical regardless of where they live — so caches keyed onasset_idare safe across machines and re-runs.provenancereuseslacing.Provenanceso the lineage chain (was_derived_from,was_generated_by) is the same for artifacts and annotations. An annotation referencing an artifact does so viaMediaRef(asset_id=artifact.asset_id, …).- classmethod from_bytes(data, *, kind, was_generated_by, was_attributed_to, path=None, url=None, was_derived_from=(), activity='create', generated_at_time=None, duration_s=None, mime=None, cost_usd=None, producer_call_id=None, rights=None)[source]#
Create an Artifact from in-memory bytes.
- Return type:
- classmethod from_path(path, *, kind, was_generated_by, was_attributed_to, was_derived_from=(), activity='create', generated_at_time=None, duration_s=None, mime=None, cost_usd=None, producer_call_id=None, rights=None)[source]#
Create an Artifact from a local file. Hashes the file’s bytes.
- Return type:
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict][pydantic.config.ConfigDict].
- to_media_ref(interval)[source]#
Return a
lacing.MediaRefpointing at this artifact.Use this to attach an annotation to a region of the artifact:
MediaRef(asset_id=artifact.asset_id, interval=…).- Return type:
MediaRef
- class lacing.ArtifactStore(catalog, blobs=None)[source]#
Bases:
MutableMappingFacade over an artifact
catalogand an optionalblobsstore.The object is a
MutableMapping[str, record]over the catalog —store[artifact_id], iteration,len,get,clearand the rest of the mapping surface all act on artifact metadata records. The heavier byte operations (put_blob(),get_blob(),has_blob()) are rich methods that are deliberately not squeezed into the mapping protocol.- Parameters:
catalog (
MutableMapping[str,BaseModel]) – Injectedid -> recordstore. Records are pydantic models (lacing.Artifactby default, but anyBaseModelworks — the store does not inspect the record’s shape).blobs (
MutableMapping[str,bytes] |None) – Injectedcontent_hash -> bytesstore, orNonefor a catalog-only store (Stage-1 metadata persistence). Blob methods raise / no-op when it isNone. This is the raw backend: on a filesystem backend its owndeland iteration do no containment check, so go throughdelete_blob()anditer_blobs()(and the read methods) rather than touchingstore.blobsdirectly (lacing#55).
Construct one with
in_memory()orfrom_directory()rather than wiring the backing stores by hand, unless you are injecting a custom backend.- blob_location(content_hash)[source]#
Resolve the cheapest servable location for a blob — without reading its bytes. The capability the HTTP layer probes to serve a blob the most efficient way, generalizing
blob_path()so filesystem / S3 / R2 backends all answer one probe:object-store backends (S3/R2) that expose a presigned-URL capability — a
url_for(content_hash)callable — return a **URL string**, so the caller can 302-redirect and let the object store serve the bytes (and HTTPRange) directly, off the app process;filesystem backends return a local Path (see
blob_path()) so the caller hands the OS the file (Range free);everything else (plain
dict, or a missing blob) returnsNone— the cue to fall back toiter_blob()streaming.
Callers treat
Noneas “stream it”, not as an error.
- blob_path(content_hash)[source]#
Return the local filesystem path of the blob, or
None.The store’s blob backend is opaque (any
MutableMapping), but some backends — notably the filesystem-backeddol.Filesproduced byfrom_directory()— store each blob as one file under a known root directory. This method exposes that path when available, so a caller (e.g. a FastAPI route serving video) can hand the OS the file descriptor and let it answer HTTPRangerequests directly. It returnsNonefor:blob stores without a
rootdirattribute (e.g. plaindict, object-store backends — callers should fall back toiter_blob());blobs that are not present.
a
content_hashthat does not resolve to a path insiderootdir(path traversal, e.g."../../etc/passwd"or an absolute path, or a same-named symlink pointing outside the store) — refused rather than served (lacing#50).
Callers must treat
Noneas the cue to use the streaming read path, not as an error. Deleting and listing blobs have contained counterparts too (delete_blob(),iter_blobs()), and thefrom_directory()catalog confines artifact ids the same way (lacing#55).The containment check is point-in-time: the returned path is the fully resolved, symlink-free location of a regular file that was inside
rootdirwhen checked. The caller opens it later, by name, so if untrusted parties can write intorootdiritself, open it withO_NOFOLLOW— or serve viaiter_blob(), which reads through a descriptor and has no such window.
- count_refs(content_hash)[source]#
How many catalog records point at
content_hash(a blob).The reference count that a garbage collector needs: a blob is safe to delete only when no catalog record still names its content hash. Content-addressed blobs are deduplicated, so two artifacts can share one blob — deleting the blob the moment one of them goes away would orphan the other.
This is the probe half of a capability pair (mirroring
blob_location()probing a blob store forurl_for): a catalog backend that can answer the count cheaply — the SQL catalog fromfrom_sql(), via an indexedcontent_hashcolumn — exposes arefcount_by_content_hashcallable, and this method uses it so the count is one indexed query, not a full catalog scan. Any other catalog (dict,Files) falls back to scanning every record. The facade is unchanged either way; only the speed differs.
- delete_blob(content_hash)[source]#
Delete the blob
content_hash;KeyErrorif there is none.The contained counterpart of
del store.blobs[content_hash](lacing#55). On a filesystem-backed store (one exposingrootdir) a key is deleted only ifhas_blob()would say it exists — a regular file that resolves insiderootdir(see_ContainedDirStore, the same gate the catalog offrom_directory()uses). A key that escapes (.., an absolute path, a NUL, a symlink pointing outside) raisesKeyErrorexactly like a missing blob, as does any key with a..segment, and nothing outside the root is touched. The deletion itself is delegated to the backend’s own__delitem__on the key as given (normalised), so the backend’s delete policy (e.g.dol.Filesmoving files to the trash) is unchanged, and deleting an in-root symlink removes the link, never the blob it points at.The check is point-in-time, like
blob_path(): a party that can write intorootdirand swap a directory component of a nested key for a symlink between the check and the delete can still redirect it. Flat content-hash keys, which is all this store writes, have no such component.Backends without a
rootdir(dict, object stores) have no filesystem to escape; the key passes through to theirdel.
- classmethod from_aws(bucket_name, uri, *, record_type=<class 'lacing.artifact.Artifact'>, collection_name='artifact_catalog', prefix=None, s3_kwargs=None, sql_kwargs=None)[source]#
The production pairing: S3-compatible blobs + SQL catalog.
A thin convenience over
from_s3()(blobs) andfrom_sql()(catalog) so the common cloud deployment is one call. The blob store is anS3Store(AWS S3 / Cloudflare R2 / MinIO / Supabase); the catalog is the durable, queryable SQL table — Postgres in production, SQLite for a smoke test. Vendor specifics live only in the two kwargs dicts.- Parameters:
bucket_name (
str) – Object-store bucket for the content-addressed blobs.uri (
str) – SQLAlchemy URI for the catalog (postgresql://…in prod).record_type (
type[BaseModel]) – Pydantic model the catalog (de)serializes.collection_name (
str) – SQL table name for the catalog.prefix (
str|None) – Optional key prefix within the bucket.s3_kwargs (
dict|None) – Extra kwargs forwarded tofrom_s3()(credentials,endpoint_urlfor R2/MinIO/Supabase,region_name, …).sql_kwargs (
dict|None) – Extra kwargs forwarded tofrom_sql()(content_hash_of, SQLAlchemy engine kwargs, …).
- Return type:
- classmethod from_directory(root, *, record_type=<class 'lacing.artifact.Artifact'>)[source]#
An ArtifactStore persisted under
root.Lays out two subdirectories:
catalog/(one<id>.jsonfile per record) andblobs/(one file per content hash). Both aredolfilesystem stores, so the same facade works unchanged over any otherdolbackend (object storage, etc.) when injected directly.Artifact ids are confined to
catalog/(lacing#55): an id whose<id>.jsonwould resolve outside it —..segments, an absolute path, a NUL, or a symlink pointing out — is refused withKeyErroron get, save and delete, and is simply notinthe store. Nested ids ("a/b") that stay inside keep working, and listing never follows a symlinked directory out ofcatalog/. See_ContainedDirStore.- Parameters:
- Return type:
- classmethod from_s3(bucket_name, *, catalog=None, prefix=None, **s3_kwargs)[source]#
An ArtifactStore with blobs in an S3-compatible object store (AWS S3 / Cloudflare R2 / MinIO / Supabase) via
s3dol.Content-addressed blobs are a natural fit for object storage: flat, immutable, dedup-friendly, forever-cacheable keys.
blob_location()returns a presigned GET URL (via s3dol’surl_for) so a serving layer can 302-redirect and let the store deliver the bytes (and HTTPRange) directly, off the app process.The blob and catalog backends are independent by design: the catalog defaults to an in-memory dict here — swap in a durable catalog (e.g. a Postgres-backed
MutableMapping) for production.- Parameters:
bucket_name (
str) – The object-store bucket.catalog (
MutableMapping[str,BaseModel] |None) – Theid -> recordcatalogMutableMapping. Defaults to{}(in-memory).prefix (
str|None) – Optional key prefix within the bucket (e.g."blobs").**s3_kwargs – Forwarded to
s3dol.s3_store—endpoint_url(set for R2 / MinIO / Supabase),region_name,profile,credentials,preset,anon,on_missing_bucket. The pre-v1 spellings (aws_access_key_id/aws_secret_access_key/aws_session_token/profile_name/make_bucket) are still accepted and translated, with aDeprecationWarning.
- Return type:
Requires
s3dol>=1(andboto3) — imported lazily so the dependency is only needed when this constructor is used.
- classmethod from_sql(uri, *, blobs=None, record_type=<class 'lacing.artifact.Artifact'>, collection_name='artifact_catalog', content_hash_of=None, **db_kwargs)[source]#
An ArtifactStore with a SQL-backed catalog (durable, queryable) via
sqldol— the SQL counterpart offrom_s3()’s object store.The catalog row schema is deliberately minimal and vendor-neutral: one
TEXTcolumn holds the record serialized exactly asfrom_directory()serializes it (record.model_dump_jsonout,record_type.model_validate_jsonin), and one indexedcontent_hashcolumn carries the record’s blob hash so the GC reference count (count_refs()) is a single indexed query rather than a full scan.Because the connection is just a SQLAlchemy URI, the same code runs on SQLite for tests and on Postgres in production — that is the whole point of the facade. Pick the backend with the
urialone:ArtifactStore.from_sql("sqlite:///artifacts.db") # local / tests ArtifactStore.from_sql("postgresql://u:p@host:5432/db") # production
The catalog and blob backends are independent: pass
blobsto pair a SQL catalog with any blob store (e.g.from_s3’sS3Store), or leave itNonefor a catalog-only store (Stage-1 metadata persistence; seefrom_aws()for the common S3 + SQL pairing).- Parameters:
uri (
str) – SQLAlchemy connection URI (sqlite:///…orpostgresql://…). The single knob that selects the vendor.blobs (
MutableMapping[str,bytes] |None) – Optionalcontent_hash -> bytesblob store.Nonefor a catalog-only store.record_type (
type[BaseModel]) – The pydantic model the catalog deserializes JSON into. Defaults toArtifact; callers with their own record schema pass their model here.collection_name (
str) – The SQL table name for the catalog.content_hash_of (
Callable[[BaseModel],str|None] |None) – How to read a record’s blob hash for the indexedcontent_hashcolumn. Defaults toasset_id(the canonical Artifact field), thencontent_hash. Pass a callable for a record whose hash lives elsewhere; records without a hash store an empty string.**db_kwargs – Forwarded to
sqldol.SQLAlchemyStore/ SQLAlchemy’screate_engine(e.g.connect_args,pool_size).
- Return type:
Requires
sqldol(andSQLAlchemy) — imported lazily so the dependency is only needed when this constructor is used.
- get_blob(content_hash)[source]#
Return the bytes for
content_hash, orNoneif absent.On a filesystem-backed store (one exposing
rootdir) the bytes are read through_open_within(), never by re-opening the path by name, so a key that would land outsiderootdir— by.., an absolute path, or a symlink, including one swapped in after the containment check (lacing#50) — reads as absent.
- has_blob(content_hash)[source]#
Whether the blob store holds
content_hash.On a filesystem-backed store this is exactly “would
get_blob()return bytes”:Falsefor a key that resolves outsiderootdiror names anything but a regular file — see_open_within(). Backends without arootdir(dict, object stores) have no filesystem to escape, so their keys pass through unfiltered.- Return type:
- classmethod in_memory()[source]#
An ArtifactStore backed entirely by in-memory dicts.
For tests, scratch work, and as the trivial reference backend. Nothing persists across processes.
- Return type:
- iter_blob(content_hash, *, chunk_size=65536)[source]#
Yield the blob’s bytes in
chunk_sizechunks.The streaming counterpart to
get_blob()— what an HTTP response body iterates over when serving a large blob without holding it all in process memory. The default implementation reads the whole blob viaget_blob()and re-chunks it; a filesystem-backed store can be swapped for a true streaming reader without changing this API.
- iter_blobs()[source]#
Yield the content hash (key) of every blob in the blob store.
The contained counterpart of
iter(store.blobs)(lacing#55). On a filesystem-backed store it yields exactly the keyshas_blob()accepts — regular files insiderootdir— and never descends into a symlinked directory (dol.Filesdoes, so its listing can yield files that live outside the root, or loop on a symlink cycle). Hidden entries are skipped, asdol.Filesskips them, which also keeps the in-flight.blob-*.partspool files ofput_blob_stream()out of the listing.Yields nothing when no blob store is configured; other backends are iterated as they are.
- put_blob(data)[source]#
Store
datacontent-addressed; return its content hash.Idempotent: identical bytes always map to the same hash and overwrite an identical blob. Delegates to
put_blob_stream()so there is exactly one write path — and therefore one atomicity story (lacing#25).- Raises:
RuntimeError – no blob store is configured.
- Return type:
- put_blob_stream(chunks)[source]#
Stream
chunkscontent-addressed; return their SHA-256 hash.The streaming-friendly counterpart to
put_blob()— callers hand in an iterable (e.g.requests.Response.iter_content) instead of materializing the whole bytestring upfront. The hash is computed on the fly.When the blob store is filesystem-backed (exposes
rootdir, as the default directory store does), the bytes spool straight to a same-directory tempfile and are renamed into place withos.replaceonce the hash is known (lacing#25). Consequences a caller may rely on:peak memory is one chunk, not 2× the payload — 100 MB videos flow through without a 200 MB spike;
a partial blob is never observable under its content address — the name a reader could find only exists after the rename, which is atomic on POSIX and on Windows for same-directory renames — so
has_blob(h) == Truereally does mean the full bytes are there;a failed or abandoned stream leaves nothing behind (the tempfile is unlinked).
Backends without a
rootdir(plaindict, object-store mappings) fall back to buffering the payload and assigning it whole — their__setitem__is the atomicity story there.- Raises:
RuntimeError – no blob store is configured.
- Return type:
- save(artifact_id, record, *, data=None)[source]#
Persist one artifact, optionally with its bytes.
Writes the blob (if
datais given) before the catalog row, so a crash never leaves the catalog pointing at missing bytes. Idempotent onartifact_id— and, for the blob, on content — so retries are safe.- Parameters:
artifact_id (
str) – The stable string id this artifact is filed under.record (
BaseModel) – The metadata record to store in the catalog.data (
bytes|None) – Optional raw bytes. When given, they are stored content-addressed and the content hash is returned; the caller is responsible for also recording that hash onrecord.
- Return type:
- Returns:
The blob’s content hash if
datawas written, elseNone.- Raises:
RuntimeError –
datawas given but no blob store is configured.
- exception lacing.BodySchemaError[source]#
Bases:
ValueErrorRaised when a body fails validation against its registered schema.
- exception lacing.EmptySchemaRegistryError[source]#
Bases:
RuntimeErrorRaised when an export would write an empty registry over real artifacts.
Almost always means the caller forgot
import lacing.bodies: the registry is populated by importing the body modules, so exporting without that import silently truncatesindex.jsonto{}and leaves the committedv<N>.jsonfiles orphaned.
- class lacing.InMemoryOpLog[source]#
Bases:
objectSimple list-backed op-log. Thread-safe via an RLock.
- class lacing.IntervalAnnotationStore(*args, **kwargs)[source]#
Bases:
ProtocolProtocol for any interval-keyed annotation store.
Conceptually a
MutableMapping[TimeInterval, list[Annotation]]: keys areTimeInterval; values are lists because multiple annotations can share an interval (different tiers, multiple annotators, soft labels).We use
Protocolrather than inheriting fromMutableMappingso backends (in-memory, SQLite, Postgres) can structurally conform without forcing a single class hierarchy. The mapping methods below match theMutableMappingABC; concrete backends likeMemoryStoreimplement the full interface.- contains(query)[source]#
Annotations whose interval strictly contains
query(Allendi).- Return type:
- extend(annotations)[source]#
Add many; equivalent to repeated
.addbut adapters can optimize.- Return type:
- lock(key)[source]#
Exclusive lock named
key, held for thewithblock.Scoped to this store’s data (for
PostgresStore, its owner and project): holders of the same key exclude each other, different keys do not. The reach of the exclusion is the backend’s:MemoryStoreexcludes threads of this process,SqliteStorealso other processes on the host,PostgresStoreevery client of the database, on any host. A holder that dies releases the lock. Not re-entrant.- Return type:
- relate(query, relations)[source]#
Annotations whose interval has any of the named
relationstoquery.Generic dispatch — useful when relations are computed at runtime.
- Return type:
- remove(annotation_id)[source]#
Remove and return the annotation with this id, or None if absent.
- Return type:
- tiers()[source]#
Registered tiers. Annotations may reference tiers not yet registered; callers decide whether that’s an error.
- update(annotation)[source]#
Replace the stored annotation that has
annotation.id, atomically.One commit: a concurrent reader sees the old annotation or the new one, never neither (
remove(id)thenadd(...)leaves a gap in between). The replacement may move to a different interval or tier. RaisesKeyErrorwhen no annotation has that id — useadd()for a new one.- Return type:
- exception lacing.LossyTimeConversionError[source]#
Bases:
ValueErrorRaised when a rate or seconds conversion would lose precision.
- class lacing.MappingStore(mapping, *, codec=(<function <lambda>>, <function <lambda>>), tiers_key='__lacing_tiers__')[source]#
Bases:
MemoryStoreAn
IntervalAnnotationStorepersisted to an injectedMutableMapping.>>> from lacing.store import MappingStore >>> backing = {} >>> store = MappingStore(backing) >>> len(backing) # nothing written until something is added 0
Reopening over the same mapping sees the same annotations and tiers.
Raises
ValueErroron a duplicate annotation id (asSqliteStoredoes) and when an annotation is assigned under a key that is not its own interval; both would otherwise not survive a reload.
- class lacing.MediaRef(**data)[source]#
Bases:
SchemaModelReference to a region of a content-addressed media asset.
- class lacing.MemoryStore[source]#
Bases:
objectIntervalAnnotationStoreimplementation overintervaltree.Conforms to the protocol in
lacing.store.base. We don’t formally inherit fromIntervalAnnotationStorebecause it’s aProtocolwith method bodies — structural typing is enough.
- exception lacing.MigrationError[source]#
Bases:
RuntimeErrorRaised when a migration step is missing or fails.
- class lacing.NodeRef(**data)[source]#
Bases:
SchemaModelReference to a node in a structured scene/document graph.
- exception lacing.NonStringBodyKeyError[source]#
Bases:
TypeErrorAn annotation
bodycontains a mapping key that is not astr.JSON object keys are strings, so
model_dump(mode="json")coerces non-string keys — and two distinct keys can coerce to the same string, silently annihilating an entry.{1: "a", "1": "b"}dumps to{"1": "b"}; a body differing only in the lost entry would digest identically, which is a wrong cache hit.Since lacing#24 the envelope refuses such a body at validation — the producer-side fix. This error lives here rather than in
lacing.modelbecause this module is deliberately import-light (stdlib only, pinned by test) and the model can import from it, not vice versa.
- class lacing.OpLog(*args, **kwargs)[source]#
Bases:
ProtocolAppend-only log of mutations.
Implementations must guarantee that the clock returned by
append()is strictly greater than every previously-returned clock value across the lifetime of the log.- append(operation, *, target_id=None, payload=None, actor='anonymous')[source]#
Append an entry; return its assigned clock.
- Return type:
- class lacing.OpLogEntry(clock, operation, target_id, payload, actor='anonymous', received_at=<factory>)[source]#
Bases:
objectOne row of the op-log.
- exception lacing.ProcessorError[source]#
Bases:
RuntimeErrorRaised when a registered processor’s invocation fails.
- class lacing.Provenance(**data)[source]#
Bases:
SchemaModelW3C PROV-O subset, embedded inline on every annotation.
- property generated_at_is_known: bool#
Falsewhengenerated_at_timeis theUNKNOWN_GENERATED_ATsentinel.The one test every freshness or ordering consumer should make before comparing
generated_at_timevalues: an unknown time cannot be ordered against a known one, and a consumer that compares anyway reads the row as older than everything (lacing#44).>>> from lacing.time import RationalTime >>> kw = dict(was_generated_by="user:x", was_attributed_to="x") >>> Provenance(generated_at_time=RationalTime(0, 1000), **kw).generated_at_is_known False >>> Provenance(generated_at_time=RationalTime.now(), **kw).generated_at_is_known True
- class lacing.RationalTime(value, rate=24000)[source]#
Bases:
objectA point in time as
value / rateseconds.Immutable. Two
RationalTimevalues with different rates compare via their rational value, soRationalTime(24, 24) == RationalTime(1, 1).Examples
>>> RationalTime(24000) == RationalTime(1, 1) True >>> RationalTime.from_seconds("1.5", rate=2).value 3
- classmethod from_seconds(seconds, rate=24000)[source]#
Build from seconds. Quantizes to
rate; raises if lossy.secondsmay be astrlike"1.001"to avoid float ingestion.- Return type:
- classmethod from_seconds_lossy(seconds, *, rate=24000, mode='round')[source]#
Build from seconds, quantizing to the nearest sample at
rate.Unlike
from_seconds()— which raisesLossyTimeConversionErrorwhen the value cannot be represented exactly — this method always succeeds by quantizing.modeselects the rounding rule:"round"— nearest sample, ties to even (default)"floor"— largest sample <=seconds"ceil"— smallest sample >=seconds
Use this when sample-level quantization is knowingly acceptable — the common case for user-supplied durations. Use
from_seconds()when exactness matters and a lossy conversion should be an error.Examples
>>> RationalTime.from_seconds_lossy("0.1", rate=3).value 0 >>> RationalTime.from_seconds_lossy("0.1", rate=3, mode="ceil").value 1
- Return type:
- classmethod from_wire(d)[source]#
Build from the wire form. Unknown keys are an error, not ignored.
RATIONAL_TIME_JSON_SCHEMAsetsadditionalProperties: falseand lacing-ui’s Zod mirror is.strict(); accepting extras here would make Python the one lax end of a contract both other ends enforce, so a{"v": 0, "r": 1, "seconds": 0.0}payload would round-trip through Python and then be rejected by the frontend.- Return type:
- classmethod now(rate=24000)[source]#
Wall-clock time as a
RationalTime, quantized torate.Uses
time.time_ns()and builds the value directly, sidestepping the float-quantization landmine offrom_seconds(float). Every producer of an annotation or artifact needs this forProvenance.generated_at_time— the only value that field reads as a known time. Tick 0 there is the UNKNOWN sentinel (lacing.model.UNKNOWN_GENERATED_AT), never the epoch.- Return type:
- to_rate(new_rate)[source]#
Re-express at
new_rate. RaisesLossyTimeConversionErroron loss.- Return type:
- class lacing.Rights(**data)[source]#
Bases:
BaseModelWho owns the bytes of an acquired artifact, and on what terms.
The fields a Creative-Commons-family licence needs to discharge its attribution duty (title, author, source, licence: “TASL”), named literal-for-literal after the two records the federation already keeps —
illustration.ImageResultandan.ir.assets.AssetSource— so both map onto this one without a rename table (provider,id,license,license_url,attribution,source_page_url,author,author_url,cacheable).Semantics worth the trap they avoid:
``Artifact.rights is None`` means “we made this”, not “unknown”. An acquired artifact whose terms nobody recorded is a
Rightswith onlyproviderset.``license is None`` means UNKNOWN, never unencumbered. Free of obligations is stated, e.g.
license="cc0-1.0".``cacheable is None`` means not stated;
Falseforbids keeping the bytes,Truepermits it.licenseis an SPDX id ("CC-BY-4.0") or a provider’s own code ("by-sa"); lacing records it verbatim and never classifies it (classification lives with the consumer, e.g.an’s licence classes).The record describes this artifact’s own bytes. A render that embeds a third-party image carries no
Rightsof its own; its obligations travel throughprovenance.was_derived_fromand are rolled up by the consumer.rights is Noneon a derivative is not a clearance.
provideris required, so an emptyRights()cannot exist: an artifact either has no record (we made it) or says where it came from. Text fields areNoneor non-blank; a blank string is refused.Caveat: an artifact catalog is keyed by
asset_id(the content hash), so two records for the same bytes are one row and the last write wins. A rights-less record written after an acquired one with identical bytes replaces it, rights included. Writers that re-record bytes they already hold must carry the existingrightsforward.
- exception lacing.SchemaMismatchError[source]#
Bases:
RuntimeErrorRaised when opening a
.annotfile with an incompatible schema.
- class lacing.SqliteOpLog(path, *, check_same_thread=True)[source]#
Bases:
objectOp-log backed by a SQLite table.
Designed to share a database file with
SqliteStoreso the store snapshot + the op-log live together and survive a singlecp project.annot project.backupstep.
- class lacing.SqliteStore(path, *, check_same_thread=True, migrate=False)[source]#
Bases:
objectSQLite-backed
IntervalAnnotationStore.- Parameters:
path (
str|PathLike) – Path to the.annotfile. Use":memory:"for an ephemeral in-memory database.check_same_thread (
bool) – Forwarded tosqlite3.connect. We hold a single connection guarded by a lock; passFalsewhen sharing across threads.migrate (
bool) – Opt-in to upgrading a file written at an olderschema_versionon open, via the ladder inlacing.store.migrations. Off by default — silently rewriting someone’s file on open is worse than refusing. Every open-time schema failure — refusal or failed migration — raisesSchemaMismatchError; a failed migration chains the ladder’sStoreMigrationErroras its cause.
- lock(key)[source]#
Exclusive lock named
key: a file lock beside the database.Excludes other threads and other processes on this host that open the same
.annotfile; the kernel frees it if the holder dies. An in-memory database (":memory:") has no file, so it falls back to a process-local lock.- Return type:
- update(annotation)[source]#
Replace the row with this id (and its R*Tree entry) in one transaction.
Readers take the same lock (
_fetchall()), so on a connection shared between threads — where a reader would otherwise see this transaction’s uncommitted state — they get the old annotation or the new one, never neither (lacing#66).- Return type:
- exception lacing.StoreMigrationError[source]#
Bases:
RuntimeErrorRaised when a store migration step is missing or fails.
- class lacing.Tier(name, *, stereotype=TierStereotype.NONE, parent=None, metadata=None)[source]#
Bases:
objectA named annotation layer with optional parent and stereotype.
Tiers are pure metadata; they don’t own annotations. The store is keyed by interval, not by tier — annotations carry their tier name as a field. This matches ELAN’s TIME_ORDER indirection (see ANN-DOC §C).
- class lacing.TierStereotype(*values)[source]#
-
Constraints on how a child tier relates to its parent tier.
Names match ELAN exactly so EAF round-trips are trivial.
- INCLUDED_IN = 'INCLUDED_IN'#
Children lie within the parent but gaps between siblings are allowed.
- NONE = 'NONE'#
No parent constraint. Top-level tier.
- SYMBOLIC_ASSOCIATION = 'SYMBOLIC_ASSOCIATION'#
One-to-one association with parent; child shares parent’s interval exactly.
- SYMBOLIC_SUBDIVISION = 'SYMBOLIC_SUBDIVISION'#
Ordered subdivision; children share parent’s interval as a sequence (no times).
- TIME_SUBDIVISION = 'TIME_SUBDIVISION'#
Children fully partition the parent’s interval (no gaps, no overlap).
- class lacing.TimeInterval(start, end)[source]#
Bases:
objectA half-open interval
[start, end).start == endis a valid point annotation, not a degenerate case. Alwaysstart <= end; constructor raisesValueErrorotherwise.- property duration: RationalTime#
end - startat the same rate asstart.
- classmethod from_wire(d)[source]#
Build from the wire form. See
RationalTime.from_wire()on extras.- Return type:
- exception lacing.UnknownBodySchemaError[source]#
Bases:
KeyErrorRaised when an annotation’s body_schema_uri has no registered model.
- lacing.annotation_body_digest(annotation)[source]#
Return the SHA-256 hex digest of
{body, body_schema_uri}only.The narrow sibling of
annotation_value_digest(). It drops the entirereference— which asset / which node / *which annotation*, not merely when — plustierandconfidence. So the same caption over two different assets digests identically here, as does the same body asserted on two different tiers or at two different confidences.That is a correctness bug in any consumer that reads any of those. Reach for it only when the consumer demonstrably depends on nothing but what the annotation says; prefer
annotation_value_digest()otherwise.- Return type:
>>> from uuid import uuid4 >>> from lacing import Annotation, MediaRef, Provenance >>> from lacing import RationalTime, TimeInterval >>> def over(asset): ... return Annotation( ... id=uuid4(), tier="words", ... reference=MediaRef( ... asset_id=asset, ... interval=TimeInterval(RationalTime(0), RationalTime(24000)), ... ), ... body={"text": "hello"}, ... body_schema_uri="annot://schema/word/v1", ... provenance=Provenance( ... was_generated_by="agent:m@1", ... was_attributed_to="thor", ... generated_at_time=RationalTime.now(), ... ), ... )
Different assets, identical body digest — this is the footgun:
>>> annotation_body_digest(over("sha256:interview")) == ( ... annotation_body_digest(over("sha256:broadcast")) ... ) True >>> annotation_value_digest(over("sha256:interview")) == ( ... annotation_value_digest(over("sha256:broadcast")) ... ) False
>>> from uuid import uuid4 >>> from lacing import Annotation, MediaRef, Provenance >>> from lacing import RationalTime, TimeInterval >>> def make(interval): ... return Annotation( ... id=uuid4(), ... tier="words", ... reference=MediaRef(asset_id="sha256:abc", interval=interval), ... body={"text": "hello"}, ... body_schema_uri="annot://schema/word/v1", ... provenance=Provenance( ... was_generated_by="agent:m@1", ... was_attributed_to="thor", ... generated_at_time=RationalTime.now(), ... ), ... ) >>> early = make(TimeInterval(RationalTime(0), RationalTime(24000))) >>> late = make(TimeInterval(RationalTime(24000), RationalTime(48000))) >>> annotation_body_digest(early) == annotation_body_digest(late) True >>> annotation_value_digest(early) == annotation_value_digest(late) False
- lacing.annotation_value_digest(annotation)[source]#
Return the SHA-256 hex digest of
annotation’s value.Covers
body,body_schema_uri,tier,referenceandconfidence. Excludesidandprovenance, so a regeneration that produces identical content produces an identical digest.Use this for freshness and early cutoff. For optimistic concurrency use
lacing.server.etag.annotation_etag()instead — two digests, two jobs, and neither substitutes for the other.- Return type:
>>> from uuid import uuid4 >>> from lacing import Annotation, MediaRef, Provenance >>> from lacing import RationalTime, TimeInterval >>> def make(**kw): ... base = dict( ... id=uuid4(), ... tier="words", ... reference=MediaRef( ... asset_id="sha256:abc", ... interval=TimeInterval(RationalTime(0), RationalTime(24000)), ... ), ... body={"text": "hello"}, ... body_schema_uri="annot://schema/word/v1", ... provenance=Provenance( ... was_generated_by="agent:m@1", ... was_attributed_to="thor", ... generated_at_time=RationalTime.now(), ... ), ... ) ... base.update(kw) ... return Annotation(**base)
A regeneration — new
id, new timestamp, same content — digests the same:>>> a = make() >>> b = make(provenance=Provenance( ... was_generated_by="agent:m@1", ... was_attributed_to="thor", ... generated_at_time=RationalTime(999), ... )) >>> annotation_value_digest(a) == annotation_value_digest(b) True
A changed body does not:
>>> annotation_value_digest(make(body={"text": "goodbye"})) == ( ... annotation_value_digest(a) ... ) False
- lacing.boundary_iou(a, b)[source]#
Mean IoU between two sets of intervals via greedy best-match.
For each interval in
a, finds its highest-IoU match inb(without replacement — once abinterval is matched it’s removed from the pool). Unmatched intervals in either set contribute 0.0 to the mean.- Return type:
- Returns:
Mean IoU ∈ [0, 1]. Returns 0.0 if both sets are empty (defensible as a “no agreement to measure” baseline).
- lacing.cohen_kappa(a, b)[source]#
Cohen’s kappa for two annotators on a categorical label.
- Parameters:
- Return type:
- Returns:
κ ∈ [-1, 1]. 1 = perfect agreement, 0 = chance, negative = worse than chance.
- Raises:
ValueError – If sequences differ in length or are empty.
- Edge cases:
If only one category appears across both annotators, both observed and expected agreement are 1.0; we return 1.0 by convention.
- lacing.export_json_schemas(target_dir, *, overwrite=True, include_meta=True, allow_empty=False)[source]#
Write every registered schema as JSON files under
target_dir.Layout:
<target_dir>/<name>/v<N>.json. Returns the list of paths written, in registration order.- Parameters:
target_dir (
str|Path) – Output directory. Created if missing.overwrite (
bool) – If False, refuse to write a file that already exists.include_meta (
bool) – If True, also write a<target_dir>/index.jsonmapping every URI to its file path and the Pydantic model’s qualified name (helps the codegen pipeline).allow_empty (
bool) – If True, permit an export from an empty registry. The default refuses, because the overwhelmingly likely cause is a missingimport lacing.bodiesand the write would destroy the committed artifacts.
- Raises:
EmptySchemaRegistryError – Nothing is registered and
allow_emptyis False.- Return type:
- lacing.get_tracer(name='lacing', version=None)[source]#
Return a tracer or a no-op fallback.
- Parameters:
- Return type:
- Returns:
opentelemetry.trace.Tracerif OTel is installed, else a no-op object whosestart_as_current_span()is a context manager yielding a no-op span.
- lacing.hash_bytes(data)[source]#
Return the canonical
asset_id(SHA-256 hex) fordata.- Return type:
>>> hash_bytes(b"hello world")[:8] 'b94d27b9'
- lacing.hash_file(path, *, chunk_size=1048576)[source]#
Return the canonical
asset_id(SHA-256 hex) for the file atpath.- Return type:
- lacing.instrument_otel(app, *, tracer_name='lacing.server')#
Add OpenTelemetry instrumentation to a FastAPI app.
Wraps every request in a span; tags the span with the response’s
X-Lacing-Clockheader value (when present) aslacing.clock.No-op when OTel isn’t installed — the app is returned unchanged.
- Parameters:
app (
Any) – The FastAPI app (fromlacing.server.create_app()).tracer_name (
str) – Name passed toget_tracer().
- Return type:
- Returns:
The same app, with middleware installed if OTel is available.
- lacing.interval_iou(a, b)[source]#
Intersection-over-Union for two time intervals.
Returns 1.0 if both are equal point intervals at the same instant; 0.0 if they don’t intersect (including when only one is a point).
- Return type:
- lacing.is_otel_active()[source]#
Quick check: is OTel installed AND a TracerProvider configured?
- Return type:
- lacing.json_schema(uri)[source]#
Return the JSON Schema for the body model registered at
uri.Pydantic’s
model_json_schema()output, plus adefaultfor every zero-argumentdefault_factoryfield (seelacing.json_schema.DefaultFactoryJsonSchema), whatever base class the registered model has.- Return type:
- lacing.krippendorff_alpha(annotations, *, distance=<function _nominal_distance>)[source]#
Krippendorff’s α across any number of annotators.
- Parameters:
annotations (
Sequence[Sequence[Optional[TypeVar(T, bound=Hashable)]]]) – A list of annotators, each a sequence of labels (one per unit). UseNonefor a missing annotation by that annotator on that unit.distance (
Callable[[TypeVar(T, bound=Hashable),TypeVar(T, bound=Hashable)],float]) – Function(x, y) -> floatmeasuring disagreement between two label values. Default is the nominal (0/1) distance.
- Return type:
- Returns:
α. 1.0 = perfect agreement, 0.0 = chance.
- Raises:
ValueError – If sequences differ in length, fewer than 2 annotators given, or fewer than 2 paired observations exist.
- lacing.maybe_span(tracer, name, **attributes)[source]#
Open a span on
tracer, attachingattributesif supported.Works with both real OTel tracers and the no-op fallback.
- lacing.migrate(body, *, from_uri, to_uri)[source]#
Migrate
bodyfromfrom_uritoto_urivia registered steps.Composes single-step migrations. Raises
MigrationErrorif any step is missing.- Return type:
- lacing.migrate_annot_file(path, *, to_version=None)[source]#
Migrate a
.annotfile in place, returning(from, to)versions.to_versiondefaults to the current build’slacing.store.sqlite.SCHEMA_VERSION. Already-current files are a no-op (from == to). Each step runs in its ownBEGIN IMMEDIATEtransaction with the version re-checked under the lock, so concurrent migrators converge and an interrupted chain resumes from the last version that completed (idempotent).Raises
StoreMigrationErrorwhen the file does not exist, is not a.annotfile, a step is missing, fails, or breaks one of the runner’s in-transaction guarantees.
- lacing.register_body_schema(uri, model)[source]#
Register
modelas the validator foruri. Returnsmodel.- Return type:
type[BaseModel]
- lacing.register_migration(*, schema_name, from_version, to_version)[source]#
Register a forward migration from
v<from_version>tov<to_version>.Decorated function takes a body
dictand returns a new bodydict. Migrations must be one major-version step at a time (to_version == from_version + 1).Re-registering the same
(schema_name, from_version)pair replaces the previous entry — convenient in tests, intentional for hot-reload.
- lacing.register_processor(func=None, *, name=None)[source]#
Register a processor under
name(defaults to the function name).The function may be sync or async; we wrap sync funcs to a coroutine.
- lacing.register_store_migration(*, store_kind, from_version, to_version)[source]#
Register a forward store migration from
from_versiontoto_version.The decorated function takes the backend’s open connection and must perform every change of the step — DDL, row rewrites, and the
meta.schema_versionwrite. Steps must be one version at a time (to_version == from_version + 1); the runner chains them.Step authors: read the module docstring’s rules — no
executescript/COMMIT/ROLLBACKinside a step, preserve rowids on table rebuilds, and rebuild the interval index withrebuild_annotations_rtree()if theannotationstable was rebuilt.Re-registering the same
(store_kind, from_version)pair replaces the previous entry.
- lacing.render_artifact_exhibit(annotations, *, out_dir, formats=('html', 'pdf', 'md'), title='Artifact exhibit', image_resolver=None)[source]#
Render an annotation graph as a human-readable artifact exhibit.
Lays every annotation out as a card — body, panel images, and in-document hyperlinks to the artifacts it derives from / feeds into. HTML is authored; the PDF and Markdown derive from it.
Panel images are written once as content-addressed sibling files under
<out_dir>/images/and referenced relatively, so the HTML and Markdown stay small; the PDF embeds them and stays a single self-contained file. Theimages/directory is created only when the graph actually has images.- Parameters:
annotations (
Iterable) – the lacing annotations to exhibit (any iterable). Order is preserved — pass them chain-ordered for a document that reads top-to-bottom.out_dir (
str|Path) – directory theexhibit.{html,pdf,md}files land in.formats (
Sequence[str]) – which ofhtml/pdf/mdto write. The HTML is always built in memory (the others derive from it).title (
str) – document title.image_resolver (
Optional[Callable[[Mapping],Optional[str]]]) –image-reference → local file pathcallback. Defaults to reading the reference’s ownpathfield; a caller whose images live behind URLs passes a resolver that downloads / caches them (keeping this module media-agnostic).
- Return type:
- Returns:
The written file paths.
- Raises:
RuntimeError – when
pdf/mdis requested but its optional converter (weasyprint/dn) is not installed — the message names the install command.
- lacing.replay_oplog(log, *, until_clock=None, target_factory=None)#
Rebuild a store by replaying
logup to (and including)until_clock.- Parameters:
- Return type:
- Returns:
The rebuilt store. Operations whose payload references unknown body schemas or tier parents are still applied; the caller is responsible for any post-replay validation.
- async lacing.run_processor_async(name, *, store, oplog, **kwargs)#
Run a processor in the current event loop. For async callers.
- Return type:
- lacing.run_processor_sync(name, *, store, oplog, **kwargs)#
Run a processor synchronously and return its result.
If the processor is async, we call it via
asyncio.run(when no loop is running) or schedule and wait on it (when a loop is already active). Most callers from sync code want the former.- Return type:
- lacing.traced(tracer, span_name=None, *, record_args=False)[source]#
Decorator: wrap a function in a span on
tracer.- Parameters:
tracer (
Any) – Fromget_tracer().span_name (
str|None) – Override the span name. Default:func.__qualname__.record_args (
bool) – If True, attach (str-coerced) positional + keyword args as span attributesarg.<n>/kwarg.<name>. Off by default — args may contain large or sensitive data.
- lacing.validate_body(body, uri)#
Validate
bodyagainst the schema registered foruri.Returns the parsed Pydantic instance. Raises
BodySchemaErroron validation failure (wrapping the underlyingpydantic.ValidationError) orUnknownBodySchemaErrorif the URI isn’t registered.- Return type:
BaseModel
Modules
I/O adapter registry. |
|
Allen's 13 interval relations as pure predicates. |
|
Content-addressed artifact references. |
|
Artifact store — a metadata catalog and a blob store behind one facade. |
|
Built-in body schemas. |
|
Command-line interface for lacing. |
|
Content digests over an annotation's value — the freshness primitive. |
|
Artifact exhibit — render an annotation graph as a readable document. |
|
Annotation envelope, references, and provenance. |
|
Operation log for time-travel debug + audit. |
|
Optional OpenTelemetry instrumentation for the lacing server. |
|
Background processors — pluggable jobs that run against a store + op-log. |
|
Inter-annotator agreement and boundary metrics. |
|
Body schema registry, JSON Schema export, and migrations. |
|
Interval-keyed annotation stores. |
|
Tiers and the five ELAN stereotypes. |
|
Rational time and half-open intervals — the foundations. |
|
High-level "track-shaped" facades on top of |
|
Optional Arq integration — queue processors through Redis. |