an.library.stores

The library mall: records, versions and blobs, each an injected MutableMapping.

ADR 0005 decision 11 and design §10. Three entities, three stores:

store	key	on disk (default backend)
records	<asset_id>	library/records/<asset_id>.json
versions	<asset_id>@<vNNN>	library/versions/<asset_id>/<vNNN>.json
blobs	<sha256>	library/blobs/<aa>/<sha256>
labels	<asset_id>@<vNNN>/<hex16>	library/labels/<asset_id>/<vNNN>/<hex16>.json

labels is write-once too: the append-only statements made about an existing version after it was published (a relabel of its unchanged content, an#307), each its own document, never rewritten or deleted.

• ``dol`` stores, unlike the project mall’s hand-written folder classes: a byte store (LocalFiles) seen through dol.wrap_kvs() with a JSON codec and a key transform. Each store is replaced by injection (build_library_mall(blobs=my_s3_store)), and business logic sees only the mapping interface — the move to S3 is an injection, not a rewrite. LocalFiles rather than dol.Files because the library needs three

  things dol.Files does not give: writes that cannot tear (temp file + os.replace), an exclusive create for write-once versions, and containment (dol.Files writes a ../x key outside its root).

• ``versions`` is write-once (WriteOnce): a version is immutable, so overwriting or deleting one raises VersionExistsError — projects pin versions, and a pin that could change under them is no pin. On the folder backend the create is exclusive, so racing publishers cannot both win.

• ``blobs`` is content-addressed and undeletable (ImmutableBlobs, a dol.content.ContentAddressedStore): every file of every version once, keyed by its SHA-256; a key can never point at changed bytes, and a blob a version pins cannot be deleted from under it.

• Keys are validated twice before they become paths: by the id grammars (an.library.ids), and by LocalFiles’ containment check.

• Nothing is created until the first write.

>>> import tempfile
>>> with tempfile.TemporaryDirectory() as d:
...     lib = build_library_mall(d)
...     sorted(lib)
['blob_rights', 'blobs', 'labels', 'records', 'versions']
>>> mem = build_library_mall(records={}, versions={}, blobs={})  # all in memory
>>> ref = mem["blobs"].add(b"<svg/>")
>>> mem["blobs"][ref.item_id]
b'<svg/>'

Module Attributes

LIBRARY_STORES

The three stores every library mall holds.

Functions

build_library_mall([root, package])	The library mall of package: records, versions (write-once), blobs (CAS).
canonical_json(obj, *[, indent])	JSON with sorted keys and no locale or platform dependence.
label_key(asset_id, version, label_id)	The labels key of one statement about a version.
split_label_key(key)	(asset_id, version, label_id) of a labels key, validated.
version_key(asset_id, version)	The versions key of one version.
split_version_key(key)	(asset_id, version) of a versions key, validated.

Classes

ImmutableBlobs([store, hasher, length, field])	The content-addressed blob store, with deletion refused.
LocalFiles(root)	relative/posix/path -> bytes under one folder, with atomic, contained writes.
WriteOnce(store)	A mapping whose keys, once written, can be neither overwritten nor deleted.

Exceptions

VersionExistsError

A write-once key was written twice, or deleted.

class an.library.stores.ImmutableBlobs(store=None, *, hasher=<built-in function openssl_sha256>, length=None, field='content')

Bases: ContentAddressedStore

The content-addressed blob store, with deletion refused.

Versions are write-once and pin their files by hash, so deleting a blob would leave a published version pointing at nothing. Reclaiming unreferenced blobs is a maintenance job (a garbage collector that counts references), not a mapping operation.

>>> blobs = ImmutableBlobs({})
>>> ref = blobs.add(b"x")
>>> del blobs[ref.item_id]
Traceback (most recent call last):
VersionExistsError: ...

an.library.stores.LIBRARY_STORES: tuple[str, ...] = ('records', 'versions', 'blobs')

The three stores every library mall holds.

class an.library.stores.LocalFiles(root)

Bases: MutableMapping

relative/posix/path -> bytes under one folder, with atomic, contained writes.

The default backend of every library store:

• contained: a key that would resolve outside the folder (.., an absolute path, a drive) raises KeyError and never touches the disk;

• atomic: a write goes to a temp file in the target folder and is moved into place with os.replace, so a crash never leaves a torn document;

• create-only (create_only()): a hard link of the fully written temp file, which fails if the target exists — the primitive WriteOnce uses so two publishers cannot both create one version;

• lazy: nothing is created until the first write, so opening (or mistyping) a library never creates folders.

>>> import tempfile
>>> with tempfile.TemporaryDirectory() as d:
...     f = LocalFiles(d + "/lib")
...     f["a/b.json"] = b"{}"
...     sorted(f), f["a/b.json"]
(['a/b.json'], b'{}')

create_only(key, data)

Write key only if it does not exist — atomically, in one step.

Return type:

None

exception an.library.stores.VersionExistsError

Bases: KeyError

A write-once key was written twice, or deleted. Versions are immutable.

class an.library.stores.WriteOnce(store)

Bases: MutableMapping

A mapping whose keys, once written, can be neither overwritten nor deleted.

Wraps any injected MutableMapping. Rewriting a key with the same value is refused too: immutability is about the key, and a silent no-op would hide a caller that believes it is replacing something.

Atomicity. When the wrapped store offers create_only(key, value) — an exclusive create that raises VersionExistsError if the key exists — every write goes through it, so two publishers racing for one key cannot both win. The default folder backend has one (a hard link of a fully written temp file); an S3 backend would map it to a conditional put (If-None-Match: *). A store without it gets a check-then-set, which is correct for one writer.

>>> versions = WriteOnce({})
>>> versions["character.a@v001"] = {"x": 1}
>>> versions["character.a@v001"] = {"x": 2}
Traceback (most recent call last):
VersionExistsError: ...

an.library.stores.build_library_mall(root=None, *, package='an', **overrides)

The library mall of package: records, versions (write-once), blobs (CAS).

Return type:

dict[str, MutableMapping]

root: the package’s data root (default: resolved by

an.library.root.library_root() — root → <PKG>_HOME → the platform data folder); the stores live under <root>/library/

package: whose library this is (an, or a genre such as cutan) overrides: a store per name to inject instead of the folder default — a

dict for tests, an S3 or database mapping later. An injected versions is still made write-once and an injected blobs still content-addressed and undeletable, so injection cannot drop an invariant.

Nothing is created until the first write: opening a library (or mistyping one on a search path) leaves the disk as it was.

an.library.stores.canonical_json(obj, *, indent=None)

JSON with sorted keys and no locale or platform dependence.

With indent=None it is the compact form hashed for a manifest; with an indent it is the on-disk form (same content, readable).

Return type:

str

>>> canonical_json({"b": 1, "a": [1, 2]})
'{"a":[1,2],"b":1}'

an.library.stores.label_key(asset_id, version, label_id)

The labels key of one statement about a version.

Return type:

str

>>> label_key("character.alice", "v002", "0123456789abcdef")
'character.alice@v002/0123456789abcdef'

an.library.stores.split_label_key(key)

(asset_id, version, label_id) of a labels key, validated.

Return type:

tuple[str, str, str]

>>> split_label_key("character.alice@v002/0123456789abcdef")
('character.alice', 'v002', '0123456789abcdef')

an.library.stores.split_version_key(key)

(asset_id, version) of a versions key, validated.

Return type:

tuple[str, str]

>>> split_version_key("character.alice@v002")
('character.alice', 'v002')

an.library.stores.version_key(asset_id, version)

The versions key of one version.

Return type:

str

>>> version_key("character.alice", "v002")
'character.alice@v002'