Durable metadata index backed by persistent-sorted-set + konserve.
Provides O(log n) lookup of commit generations by custom metadata keys, with lazy loading from disk (O(1) startup) and incremental updates.
Each entry is {:branch b :key k :value v :generation g}. Sorted by [:branch :key :value] enabling efficient exact and floor queries.
Every store call is {:sync? true}. It used to block on konserve's async
channel with <!!, which measured 8.6x slower per write on the same
filestore — a flush issues eight of them, so it dominated commit cost for any
index carrying metadata. scriptum.konserve was already synchronous
throughout; this namespace was the outlier.
Durable metadata index backed by persistent-sorted-set + konserve.
Provides O(log n) lookup of commit generations by custom metadata keys,
with lazy loading from disk (O(1) startup) and incremental updates.
Each entry is {:branch b :key k :value v :generation g}.
Sorted by [:branch :key :value] enabling efficient exact and floor queries.
Every store call is `{:sync? true}`. It used to block on konserve's async
channel with `<!!`, which measured 8.6x slower per write on the same
filestore — a flush issues eight of them, so it dominated commit cost for any
index carrying metadata. `scriptum.konserve` was already synchronous
throughout; this namespace was the outlier.(close-index! mi)Flush and close the metadata index store.
Flush and close the metadata index store.
(create-metadata-index base-path)Create or restore a metadata index at base-path/scriptum-metadata/. Returns a MetadataIndex record.
Create or restore a metadata index at base-path/scriptum-metadata/. Returns a MetadataIndex record.
(find-exact mi branch key value)Find exact match for branch+key+value. Returns {:generation n} or nil.
Find exact match for branch+key+value. Returns {:generation n} or nil.
(find-floor mi branch key value)Find floor match: latest entry where value <= target for branch+key. Returns {:generation n :indexed-value v} or nil.
Find floor match: latest entry where value <= target for branch+key.
Returns {:generation n :indexed-value v} or nil.(flush-index! mi)Persist the PSS index to konserve.
Persist the PSS index to konserve.
(index! mi branch-name custom-metadata generation)Index custom metadata entries for a commit. branch-name: string, custom-metadata: map of key->value strings, generation: long.
Index custom metadata entries for a commit. branch-name: string, custom-metadata: map of key->value strings, generation: long.
(mark kv-store)Every key this index needs kept, for a collector over its store.
MUST BE UNIONED WITH scriptum.konserve/mark WHEN BOTH SHARE A STORE.
konserve.gc/sweep! is allow-list — it deletes every key not named — and
scriptum's own mark cannot infer these: :metadata/roots and
:metadata/freed are bare keywords, and every PSS node is stored under a
raw (random-uuid), so there is no [:scriptum …] prefix to match on. The
result was not a collision but something quieter: a sweep from scriptum's
whitelist deleted the roots and every node, leaving the in-memory atom as the
only surviving copy until restart.
No in-tree caller shares a store today — create-metadata-index always builds
its own filestore under <path>/scriptum-metadata — but open-store-index
accepts a :metadata-index over any store, so the wiring is one line away.
Walks the tree from the roots, so it costs a read per node.
Every key this index needs kept, for a collector over its store. MUST BE UNIONED WITH `scriptum.konserve/mark` WHEN BOTH SHARE A STORE. `konserve.gc/sweep!` is allow-list — it deletes every key not named — and scriptum's own mark cannot infer these: `:metadata/roots` and `:metadata/freed` are bare keywords, and every PSS node is stored under a raw `(random-uuid)`, so there is no `[:scriptum …]` prefix to match on. The result was not a collision but something quieter: a sweep from scriptum's whitelist deleted the roots and every node, leaving the in-memory atom as the only surviving copy until restart. No in-tree caller shares a store today — `create-metadata-index` always builds its own filestore under `<path>/scriptum-metadata` — but `open-store-index` accepts a `:metadata-index` over any store, so the wiring is one line away. Walks the tree from the roots, so it costs a read per node.
Comparator for metadata entries: [:branch :key :value]
Comparator for metadata entries: [:branch :key :value]
(rebuild-from-snapshots! mi snapshots-by-branch)Rebuild the metadata index for the branches named in snapshots-by-branch.
snapshots-by-branch: map of branch-name -> seq of snapshot maps (each with :custom-metadata and :generation).
AUTHORITATIVE ONLY FOR THE BRANCHES IT IS GIVEN. Entries for any other branch
are carried across untouched. It used to rebuild from the map alone and
reset! the result, so a branch missing from it lost every entry — and the
caller drops exactly the branches it could not read, which for scriptum.core/gc!
means any branch with a live writer, i.e. the ordinary main+feature workflow.
A collection on main silently erased the feature branch's metadata.
Absence is not evidence here: a branch that could not be re-derived is a branch we know nothing about, not one we know to be empty.
Rebuild the metadata index for the branches named in `snapshots-by-branch`. snapshots-by-branch: map of branch-name -> seq of snapshot maps (each with :custom-metadata and :generation). AUTHORITATIVE ONLY FOR THE BRANCHES IT IS GIVEN. Entries for any other branch are carried across untouched. It used to rebuild from the map alone and `reset!` the result, so a branch missing from it lost every entry — and the caller drops exactly the branches it could not read, which for `scriptum.core/gc!` means any branch with a live writer, i.e. the ordinary main+feature workflow. A collection on main silently erased the feature branch's metadata. Absence is not evidence here: a branch that could not be re-derived is a branch we know nothing about, not one we know to be empty.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |