lacing.store

Interval-keyed annotation stores.

Public surface: IntervalAnnotationStore (the facade), MemoryStore, MappingStore (persisted to any injected MutableMapping), SqliteStore (the .annot on-disk format), the optional PostgresStore, and the store-schema migration ladder (lacing.store.migrations).

Functions

register_store_migration(*, store_kind, ...)	Register a forward store migration from from_version to to_version.
migrate_annot_file(path, *[, to_version])	Migrate a .annot file in place, returning (from, to) versions.
reachable_versions(store_kind, from_version)	Versions reachable from from_version by chaining registered steps.
rebuild_annotations_rtree(conn)	Rebuild the interval index from the annotations table, in place.
get_pool(connection_string, **pool_kwargs)	Return a process-wide psycopg_pool.ConnectionPool for a conninfo.
close_all_pools()	Close every registered connection pool (test teardown / shutdown).

Classes

IntervalAnnotationStore(*args, **kwargs)	Protocol for any interval-keyed annotation store.
MemoryStore()	IntervalAnnotationStore implementation over intervaltree.
MappingStore(mapping, *[, codec, tiers_key])	An IntervalAnnotationStore persisted to an injected MutableMapping.
MappingCodec(encode, decode)	How a JSON-ready dict is turned into what the mapping stores, and back.
SqliteStore(path, *[, check_same_thread, ...])	SQLite-backed IntervalAnnotationStore.
PostgresStore(connection_string, *[, rate, ...])	PostgreSQL-backed IntervalAnnotationStore, scoped to one tenant.

Exceptions

SchemaMismatchError	Raised when opening a .annot file with an incompatible schema.
StoreMigrationError	Raised when a store migration step is missing or fails.
PgSchemaMismatchError	Raised on opening a database whose schema_version differs.
TierOverlapError	Raised when an insert would violate a per-tier no-overlap constraint.
RateMismatchError	Raised when an annotation's interval cannot be represented at the project rate.

class lacing.store.IntervalAnnotationStore(*args, **kwargs)

Bases: Protocol

Protocol for any interval-keyed annotation store.

Conceptually a MutableMapping[TimeInterval, list[Annotation]]: keys are TimeInterval; values are lists because multiple annotations can share an interval (different tiers, multiple annotators, soft labels).

We use Protocol rather than inheriting from MutableMapping so backends (in-memory, SQLite, Postgres) can structurally conform without forcing a single class hierarchy. The mapping methods below match the MutableMapping ABC; concrete backends like MemoryStore implement the full interface.

add(annotation)

Append annotation to the list at its reference interval.

Return type:

None

all()

Iterate every annotation in the store, order unspecified.

Return type:

Iterator[Annotation]

at_tier(tier_name, query)

Annotations on tier_name that intersect query.

Return type:

Iterator[Annotation]

by_tier(tier_name)

All annotations on tier_name, regardless of interval.

Return type:

Iterator[Annotation]

contains(query)

Annotations whose interval strictly contains query (Allen di).

Return type:

Iterator[Annotation]

during(query)

Annotations whose interval is strictly inside query (Allen d).

Return type:

Iterator[Annotation]

equals(query)

Allen =: identical interval.

Return type:

Iterator[Annotation]

extend(annotations)

Add many; equivalent to repeated .add but adapters can optimize.

Return type:

None

finishes(query)

Allen f: later start, same end.

Return type:

Iterator[Annotation]

intersects(query)

Annotations whose interval shares any time with query.

Return type:

Iterator[Annotation]

lock(key)

Exclusive lock named key, held for the with block.

Scoped to this store’s data (for PostgresStore, its owner and project): holders of the same key exclude each other, different keys do not. The reach of the exclusion is the backend’s: MemoryStore excludes threads of this process, SqliteStore also other processes on the host, PostgresStore every client of the database, on any host. A holder that dies releases the lock. Not re-entrant.

Return type:

AbstractContextManager[None]

meets(query)

Allen m: a.end == q.start.

Return type:

Iterator[Annotation]

overlaps(query)

Strict Allen o: a.start < q.start < a.end < q.end.

Return type:

Iterator[Annotation]

relate(query, relations)

Annotations whose interval has any of the named relations to query.

Generic dispatch — useful when relations are computed at runtime.

Return type:

Iterator[Annotation]

remove(annotation_id)

Remove and return the annotation with this id, or None if absent.

Return type:

Annotation | None

starts(query)

Allen s: same start, earlier end.

Return type:

Iterator[Annotation]

tiers()

Registered tiers. Annotations may reference tiers not yet registered; callers decide whether that’s an error.

Return type:

Iterator[Tier]

update(annotation)

Replace the stored annotation that has annotation.id, atomically.

One commit: a concurrent reader sees the old annotation or the new one, never neither (remove(id) then add(...) leaves a gap in between). The replacement may move to a different interval or tier. Raises KeyError when no annotation has that id — use add() for a new one.

Return type:

None

class lacing.store.MappingCodec(encode, decode)

Bases: NamedTuple

How a JSON-ready dict is turned into what the mapping stores, and back.

decode: Callable[[Any], dict]

Alias for field number 1

encode: Callable[[dict], Any]

Alias for field number 0

class lacing.store.MappingStore(mapping, *, codec=(<function <lambda>>, <function <lambda>>), tiers_key='__lacing_tiers__')

Bases: MemoryStore

An IntervalAnnotationStore persisted to an injected MutableMapping.

>>> from lacing.store import MappingStore
>>> backing = {}
>>> store = MappingStore(backing)
>>> len(backing)  # nothing written until something is added
0

Reopening over the same mapping sees the same annotations and tiers.

Raises ValueError on a duplicate annotation id (as SqliteStore does) and when an annotation is assigned under a key that is not its own interval; both would otherwise not survive a reload.

close()

No-op. Writes go through immediately and the mapping is the caller’s.

Present so code that calls store.close() after each use (as it does for SqliteStore) can treat both alike; the store stays usable.

Return type:

None

class lacing.store.MemoryStore

Bases: object

IntervalAnnotationStore implementation over intervaltree.

Conforms to the protocol in lacing.store.base. We don’t formally inherit from IntervalAnnotationStore because it’s a Protocol with method bodies — structural typing is enough.

lock(key)

Exclusive lock named key among the threads of this process.

Return type:

AbstractContextManager[None]

exception lacing.store.PgSchemaMismatchError

Bases: RuntimeError

Raised on opening a database whose schema_version differs.

class lacing.store.PostgresStore(connection_string, *, rate=24000, owner_id='default', project_id='default', autocommit=True, use_pool=True)

Bases: object

PostgreSQL-backed IntervalAnnotationStore, scoped to one tenant.

A store instance is bound to one (owner_id, project_id) pair; multiple instances over the same database (different project ids) coexist without seeing each other’s annotations or tiers (see the module docstring on multi-tenancy). The mapping/Allen/tier surface is unchanged — the tenant scoping is transparent.

Parameters:

• connection_string (str | dict) – A psycopg-compatible conninfo URL, or a kwargs dict (host/port/user/password/dbname).

• rate (int) – Project-wide rate for this (owner, project). Set on first init for the pair; re-opening the same pair with a different rate raises PgSchemaMismatchError. Different projects in the same DB may have different rates.

• owner_id (str) – Tenant owner. Defaults to DEFAULT_OWNER_ID — the forward seam for the access layer (reelee#174); enforcement deferred.

• project_id (str) – Logical project key. Defaults to DEFAULT_PROJECT_ID. nw passes the project’s project_asset_id here.

• autocommit (bool) – If True (default), statements run in their own transaction.

• use_pool (bool) – When True (default) and the connection is given as a string, borrow a connection from a process-wide pool keyed by the conninfo (amortizes per-op connect cost; see get_pool()). Falls back to a dedicated connection when psycopg_pool is unavailable, when the connection is given as a dict, or when use_pool=False.

add_tier(tier, *, enforce_no_overlap=False)

Add or update a tier.

Parameters:

• tier (Tier) – Tier definition.

• enforce_no_overlap (bool) – If True, install a per-tier EXCLUDE USING GIST constraint forbidding overlapping annotations within this tier. Required for proper TIME_SUBDIVISION enforcement. Cannot be toggled after the tier has annotations.

Return type:

None

lock(key)

Exclusive lock named key for this (owner, project), across hosts.

A session-level pg_advisory_lock keyed on a 64-bit hash of owner_id + project_id + key, taken on a dedicated connection (never the store’s own, which other threads and transactions share) and released in finally. If the holder crashes, Postgres drops the lock when its connection dies, so nothing can go stale. Blocks until available; not re-entrant.

Return type:

Iterator[None]

update(annotation)

Replace the row with this id in ONE UPDATE statement (lacing#66).

A single statement is atomic in any isolation mode, so a concurrent reader sees the old row or the new one — unlike remove then add, which commit twice. Raises KeyError if the id is not in this (owner, project) and TierOverlapError like add().

Return type:

None

exception lacing.store.RateMismatchError

Bases: LossyTimeConversionError

Raised when an annotation’s interval cannot be represented at the project rate.

exception lacing.store.SchemaMismatchError

Bases: RuntimeError

Raised when opening a .annot file with an incompatible schema.

class lacing.store.SqliteStore(path, *, check_same_thread=True, migrate=False)

Bases: object

SQLite-backed IntervalAnnotationStore.

Parameters:

• path (str | PathLike) – Path to the .annot file. Use ":memory:" for an ephemeral in-memory database.

• check_same_thread (bool) – Forwarded to sqlite3.connect. We hold a single connection guarded by a lock; pass False when sharing across threads.

• migrate (bool) – Opt-in to upgrading a file written at an older schema_version on open, via the ladder in lacing.store.migrations. Off by default — silently rewriting someone’s file on open is worse than refusing. Every open-time schema failure — refusal or failed migration — raises SchemaMismatchError; a failed migration chains the ladder’s StoreMigrationError as its cause.

lock(key)

Exclusive lock named key: a file lock beside the database.

Excludes other threads and other processes on this host that open the same .annot file; the kernel frees it if the holder dies. An in-memory database (":memory:") has no file, so it falls back to a process-local lock.

Return type:

AbstractContextManager[None]

update(annotation)

Replace the row with this id (and its R*Tree entry) in one transaction.

Readers take the same lock (_fetchall()), so on a connection shared between threads — where a reader would otherwise see this transaction’s uncommitted state — they get the old annotation or the new one, never neither (lacing#66).

Return type:

None

exception lacing.store.StoreMigrationError

Bases: RuntimeError

Raised when a store migration step is missing or fails.

exception lacing.store.TierOverlapError

Bases: RuntimeError

Raised when an insert would violate a per-tier no-overlap constraint.

lacing.store.close_all_pools()

Close every registered connection pool (test teardown / shutdown).

Return type:

None

lacing.store.get_pool(connection_string, **pool_kwargs)

Return a process-wide psycopg_pool.ConnectionPool for a conninfo.

One pool per distinct connection_string (the keying happens here, not at the call-site), created lazily on first request. Raises ImportError if psycopg_pool is not installed — callers that want a graceful fallback catch it (see PostgresStore).

Parameters:

• connection_string (str) – psycopg conninfo URL (the registry key).

• **pool_kwargs – forwarded to ConnectionPool on first creation (e.g. min_size, max_size); ignored on reuse.

Returns:

The shared pool for this conninfo.

lacing.store.migrate_annot_file(path, *, to_version=None)

Migrate a .annot file in place, returning (from, to) versions.

to_version defaults to the current build’s lacing.store.sqlite.SCHEMA_VERSION. Already-current files are a no-op (from == to). Each step runs in its own BEGIN IMMEDIATE transaction with the version re-checked under the lock, so concurrent migrators converge and an interrupted chain resumes from the last version that completed (idempotent).

Raises StoreMigrationError when the file does not exist, is not a .annot file, a step is missing, fails, or breaks one of the runner’s in-transaction guarantees.

Return type:

tuple[int, int]

lacing.store.reachable_versions(store_kind, from_version)

Versions reachable from from_version by chaining registered steps.

Ascending, excluding from_version itself. Empty when no step leaves from_version — which is what a refusal message should say out loud instead of the bare “run a migration” it used to say.

Return type:

tuple[int, ...]

lacing.store.rebuild_annotations_rtree(conn)

Rebuild the interval index from the annotations table, in place.

For use inside a migration step after a table rebuild. Reproduces the store’s ULP-widening contract (bounds widened by one float ULP so float→exact-bound comparisons never drop hits — see lacing.store.sqlite). Returns the number of rows indexed.

Return type:

int

lacing.store.register_store_migration(*, store_kind, from_version, to_version)

Register a forward store migration from from_version to to_version.

The decorated function takes the backend’s open connection and must perform every change of the step — DDL, row rewrites, and the meta.schema_version write. Steps must be one version at a time (to_version == from_version + 1); the runner chains them.

Step authors: read the module docstring’s rules — no executescript / COMMIT / ROLLBACK inside a step, preserve rowids on table rebuilds, and rebuild the interval index with rebuild_annotations_rtree() if the annotations table was rebuilt.

Re-registering the same (store_kind, from_version) pair replaces the previous entry.

Modules

base	IntervalAnnotationStore — the headline Pythonic API.
locking	Named exclusive locks for stores: the mechanisms behind store.lock(key).
mapping	MappingStore — an IntervalAnnotationStore persisted to any MutableMapping.
memory	In-memory annotation store backed by an interval tree.
migrations	Store-level schema migrations — the on-disk counterpart of the body ladder.
postgres	PostgreSQL-backed annotation store using int8range + GiST.
sqlite	SQLite-backed annotation store and the .annot portable file format.