Liking cljdoc? Tell your friends :D

vaelii.sqlite.record-store

A SQLite target for the engine's record store seam (vaelii.impl.protocols/RecordStore) — the durable ground truth a KB is recovered from: canonical sentexes and justifications, keyed by integer handle, in a single SQLite file.

This is the other SQLite lane, and the one the snapshot sink argues around. A records store over a networked database pays a round trip per probe — two orders of magnitude off local, which is why that lane stays a snapshot. Embedded SQLite is not networked: the driver runs in-process against a file, a warm point read is single-digit microseconds (measured ~6µs against the disk store's ~3µs), so a live records backend is a real third option beside :memory and :disk — an always-current, single-file, SQL-inspectable store, where the disk backend is a bespoke mmap pair and the snapshot is a frozen image you re-export.

The shape

The seam is 17 ops (put-/get-/delete- for sentexes, justifications and provenance; next-id; the *-ids enumerations; premise marking). Here:

  • every record is one row in record (id, kind, frame, premise, strength) — the handle is the primary key, kind splits sentexes (0) from justifications (1), and frame is the whole record nippy-frozen, so a fetch thaws back type-identical (AtomicSentex stays an AtomicSentex);
  • a sentex's assumption strength rides the strength column as the authoritative value and is assoced back onto the thawed record on read — so mark-premise is a one-row column update, never a frame rewrite, and the column and frame cannot drift because the column always wins;
  • handles are never reissued. next-id walks an in-memory counter, and a monotonic high_water row (bumped inside each put's transaction, reconciled with the live max on open) survives a delete-of-the-max-handle across a reopen — the disk store gets the same guarantee from an idx file that never shrinks.

The file

Point the store at a file-backed db-spec ({:dbtype "sqlite" :dbname "kb.db"}). It holds one long-lived connection for its lifetime under WAL + synchronous=NORMAL — the disk store's durability shape (buffered writes, periodic fsync, a crash loses at most the last unsynced writes, never consistency). A JDBC connection is not thread-safe, so every op serializes on a lock, with a synchronized access-ordered LRU in front so a hot read never reaches SQLite or the lock. close/clear-records! release it. It is java.io.Closeable.

Boundary

Apache-2.0, and an adapter: it implements the SSPL engine's vaelii.impl.protocols/RecordStore and is never depended on by it. Core wires it in by a lazy requiring-resolve, the same way it reaches the dense TMS, so the engine never loads the SQLite driver unless a KB asks for this backend.

A SQLite target for the engine's **record store** seam
(`vaelii.impl.protocols/RecordStore`) — the durable ground truth a KB is
recovered from: canonical sentexes and justifications, keyed by integer handle,
in a single SQLite file.

This is the **other SQLite lane**, and the one the snapshot sink argues around.
A records store over a *networked* database pays a round trip per probe — two
orders of magnitude off local, which is why that lane stays a snapshot.
**Embedded SQLite is not networked**: the driver runs in-process against a file,
a warm point read is single-digit microseconds (measured ~6µs against the disk
store's ~3µs), so a live records backend is a real third option beside `:memory`
and `:disk` — an always-current, single-file, SQL-inspectable store, where the
disk backend is a bespoke mmap pair and the snapshot is a frozen image you
re-export.

## The shape

The seam is 17 ops (`put-`/`get-`/`delete-` for sentexes, justifications and
provenance; `next-id`; the `*-ids` enumerations; premise marking).  Here:

* every record is one row in `record (id, kind, frame, premise, strength)` — the
  handle is the primary key, `kind` splits sentexes (0) from justifications (1),
  and `frame` is the **whole record** nippy-frozen, so a fetch thaws back
  type-identical (`AtomicSentex` stays an `AtomicSentex`);
* a sentex's **assumption strength** rides the `strength` column as the
  authoritative value and is `assoc`ed back onto the thawed record on read — so
  `mark-premise` is a one-row column update, never a frame rewrite, and the
  column and frame cannot drift because the column always wins;
* **handles are never reissued.**  `next-id` walks an in-memory counter, and a
  monotonic `high_water` row (bumped inside each put's transaction, reconciled
  with the live max on open) survives a delete-of-the-max-handle across a
  reopen — the disk store gets the same guarantee from an idx file that never
  shrinks.

## The file

Point the store at a **file-backed** db-spec (`{:dbtype "sqlite" :dbname
"kb.db"}`).  It holds **one long-lived connection** for its lifetime under
WAL + `synchronous=NORMAL` — the disk store's durability shape (buffered writes,
periodic fsync, a crash loses at most the last unsynced writes, never
consistency).  A JDBC connection is not thread-safe, so every op serializes on a
lock, with a synchronized access-ordered LRU in front so a hot read never
reaches SQLite or the lock.  `close`/`clear-records!` release it.  It is
`java.io.Closeable`.

## Boundary

Apache-2.0, and an **adapter**: it implements the SSPL engine's
`vaelii.impl.protocols/RecordStore` and is never depended on by it.  Core wires
it in by a lazy `requiring-resolve`, the same way it reaches the dense TMS, so
the engine never loads the SQLite driver unless a KB asks for this backend.
raw docstring

vaelii.sqlite.snapshot

A SQLite target for the engine's snapshot seam (vaelii.impl.io.snapshot) — a SnapshotSink that writes a KB image to a single-file database and a SnapshotSource that reads it back.

This is the good SQLite lane. A live records/index store over a database pays a round trip per probe, two orders of magnitude off local, so the query path stays RAM-resident on every backend. A snapshot is the opposite shape: an image is O(sections) bulk blob transfers, not O(records) tiny probes. SQLite is where that shape is cheapest of all — the whole image is one file with no server to stand up, so "put my KB in a file" — for backup, for shipping a corpus to another host, for embedding a KB beside an app — is answered here, and the file is the artifact you copy.

The shape

The seam is two ops each side (write-section!/commit!, read-manifest/read-section) and the image is a set of named sections plus a manifest. Here:

  • a section is a row stream (image, section, seq, chunk blob) — each chunk a nippy-frozen batch of frames, so a section of a million entries is a hundred bulk inserts, not a million;
  • the whole image writes inside one transaction and the manifest row commits it — so a crash leaves no manifest, read-manifest returns nil, and the caller rebuilds. That is the file sink's "write manifest.edn last" rule, but enforced by the database rather than by ordering: a failed write rolls back the sections too, it does not leave half of them;
  • the validity stamp (kv/index-layout-version and the records fingerprint) rides both a column (queryable) and the manifest blob, and the validate-or-discard check is the shared snapshot/decision — a mismatched image is discarded and the source rebuilds, never trusted.

A section written through this sink reads back frame-identical through any source (file, memory, Postgres, SQLite) — the portability the seam exists to give.

The file

Point the sink at a file-backed db-spec ({:dbtype "sqlite" :dbname "kb.db"} or a jdbc:sqlite:… url). The sink, the source, and ensure-schema! each open their own connection, and for a file they all see the same database — the same multi-connection shape the Postgres sink has. An in-memory :memory: spec gives each connection its own database, so the schema one opens is invisible to the next; use a file, or a shared-cache datasource, if you need memory.

Boundary

Apache-2.0, and an adapter: it depends on the SSPL engine's seam and is never depended on by it. It implements vaelii.impl.io.snapshot's protocols, the same way a record-store adapter implements vaelii.impl.protocols.

A SQLite target for the engine's snapshot seam
(`vaelii.impl.io.snapshot`) — a `SnapshotSink` that writes a KB image to a
single-file database and a `SnapshotSource` that reads it back.

This is the **good SQLite lane**.  A live records/index store over a database
pays a round trip per probe, two orders of magnitude off local, so the query
path stays RAM-resident on every backend.  A **snapshot** is the opposite
shape: an image is `O(sections)` bulk blob transfers, not `O(records)` tiny
probes.  SQLite is where that shape is cheapest of all — the whole image is one
file with no server to stand up, so "put my KB in a file" — for backup, for
shipping a corpus to another host, for embedding a KB beside an app — is
answered here, and the file is the artifact you copy.

## The shape

The seam is two ops each side (`write-section!`/`commit!`,
`read-manifest`/`read-section`) and the image is a set of **named sections**
plus a **manifest**.  Here:

* a section is a row stream `(image, section, seq, chunk blob)` — each chunk a
  nippy-frozen batch of frames, so a section of a million entries is a hundred
  bulk inserts, not a million;
* the whole image writes inside **one transaction** and the **manifest row
  commits it** — so a crash leaves no manifest, `read-manifest` returns nil,
  and the caller rebuilds.  That is the file sink's "write `manifest.edn`
  last" rule, but enforced by the database rather than by ordering: a failed
  write rolls back the sections too, it does not leave half of them;
* the validity stamp (`kv/index-layout-version` and the records fingerprint)
  rides both a **column** (queryable) and the manifest blob, and the
  validate-or-discard check is the shared `snapshot/decision` — a mismatched
  image is discarded and the source rebuilds, never trusted.

A section written through this sink reads back frame-identical through any
source (file, memory, Postgres, SQLite) — the portability the seam exists to
give.

## The file

Point the sink at a **file-backed** db-spec (`{:dbtype "sqlite" :dbname
"kb.db"}` or a `jdbc:sqlite:…` url).  The sink, the source, and
`ensure-schema!` each open their own connection, and for a file they all see
the same database — the same multi-connection shape the Postgres sink has.  An
in-memory `:memory:` spec gives **each connection its own database**, so the
schema one opens is invisible to the next; use a file, or a shared-cache
datasource, if you need memory.

## Boundary

Apache-2.0, and an **adapter**: it depends on the SSPL engine's seam and is
never depended on by it.  It implements `vaelii.impl.io.snapshot`'s protocols,
the same way a record-store adapter implements `vaelii.impl.protocols`.
raw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close