vaelii.impl.memory — com.vaelii/vaelii 0.13.0

The default in-memory backends for the two storage protocols, selected at KB construction. The engine above the protocols never touches a concrete store, so a KB built on these runs the whole engine with no external dependency; the on-disk backend (vaelii.impl.disk) is the durable alternative.

The record store implements RecordStore directly over maps. The index reuses vaelii.impl.kv/KvIndexStore — the one trie/roots/index implementation — over a MemoryKvBackend: one map keyed by the structured key vectors (equal vectors are equal keys), holding a Long at each counter key and a set at each set key. kv-intersect is a clojure.set/intersection; kv-members returns the stored set by reference (no serialization, no copy).

Durable-within-the-JVM semantics by space number. Two KBs constructed over the same :space number must share state, or the persistence/recovery tests (a second KB restarted over the same databases) would find an empty store. A process-global registry keyed by space number provides that: (memory-record-store {:space 15}) twice returns records backed by one state atom, and by one handle counter beside it. clear-records! / clear-index! empty a space. (State lives only for the life of the JVM; the on-disk backend is what survives a process restart.)

Single-writer. Pure runs one writer (docs/storage.md, "The single-writer contract"), so a store's state is one atom mutated by swap!; reads deref a snapshot and are lock-free. Interleaved writers would not be serializable.

*bulk-txn* clj

During a bulk load, {:state <the loaded backend's state atom> :txn <a volatile! holding a transient of its map>}. While bound, that backend's writes land on the transient (assoc! — no per-op HAMT path copy) and the whole load is one persistent! at the end, instead of a swap! per fact. nil (the default) leaves every op on the persistent atom, so nothing outside a bulk load pays for the binding's existence — and reads go to the backing atom in both modes, so the query hot path never branches on it. The backing atom is therefore stale for the life of the load: correct only because the sole mid-load reader (note-opposed's [:false b] probe) reads the always-empty negative side, and every real read happens after the closing persistent!. Use with-bulk-writes for a positive/monotonic, distinct load; a corpus with (not …) facts must rebuild-opposed! after it (the atom the opposed set is derived from was stale during the load).

The binding names the backend it is for, because a dynamic binding is per thread, not per backend: a write this thread makes to any other MemoryKvBackend while the load runs — a second KB's index, a chaining callback asserting elsewhere — lands on that backend's own atom (txn-for), never on the loaded one's transient.

The accumulator's life is the batch's, and no longer. with-bulk-writes clears the volatile as it installs, so the binding a body conveyed somewhere — a future, a lazy seq realized afterwards — finds nothing to write on and takes the atom, which is where a write outside the batch belongs. A transient reached after its persistent! is not a slow write but a thrown one, and on a path nobody expected to be able to throw.

bulk-writes* clj

(bulk-writes* bk f)

with-bulk-writes' body as a thunk — the macro is a wrapper over this, so the accumulator's whole life is one function's local and nothing about it is spliced into a caller's code.

Three steps, in this order, and each is what makes the batch's accumulator private to the batch:

• the state map is read once, into base, and the transient is built off that value. base is what the install is checked against, so the batch knows what it was accumulating over rather than assuming;
• the volatile is cleared before the install, so the binding a body conveyed elsewhere (a future, a lazy seq realized later) finds no accumulator and takes the atom — the correct home for a write outside the batch, and not a persistent!-ed transient that would throw;
• the install is a compare-and-set against base, not a reset!. The atom is per space and shared by every index store over that space, so a reset! would silently discard anything that reached it while the batch ran — a second bulk load stacked over this one being the way to get there without two threads. Under the single-writer contract nothing does, which is exactly why the check is affordable: it is one CAS per batch, and it fires only where an overwrite would lose data.

A failed install is :stacked-batch. It is raised only when the body itself returned — a body that threw has a failure of its own to report, and replacing it with this one would hide it.

memory-index-store clj

(memory-index-store opts)

An in-memory IndexStore — KvIndexStore over a MemoryKvBackend.

memory-kv-backend clj

(memory-kv-backend {:keys [space] :or {space 0}})

An in-memory KvBackend. Only :space in opts matters (it selects the shared state atom, so two index stores over the same space number share one map).

memory-record-store clj

(memory-record-store {:keys [space] :or {space 0}})

An in-memory RecordStore. Only :space in opts matters: it selects the shared state atom and the shared handle counter, which have to be the same pair for two stores over one space or the second would re-issue handles the first had already written.

with-bulk-writes clj macro

(with-bulk-writes backend & body)

Run body with backend's index writes accumulated on one TRANSIENT of its state map, persisted back in a single step at the end — the write-side fast path for a bulk load (millions of trie assoc!s with no per-op HAMT path copy, one persistent! instead of a swap! per fact). A no-op wrapper unless backend is a MemoryKvBackend, so a non-memory store (disk) just runs body on its own batched path. See *bulk-txn* for the read-staleness contract: a positive/monotonic, distinct load only; a corpus with (not …) facts must rebuild-opposed! after; and bulk-writes*, which this delegates to, for what keeps the accumulator the batch's own.