vaelii-sqlite

SQLite targets for vaelii's storage seams — an Apache-2.0 adapter (com.vaelii/sqlite) on the SSPL engine. It depends on core; core never depends on it.

What's here

Two SQLite lanes over the engine's storage seams — a snapshot sink (a KB image in a file) and a record store (a live durable backend). Pick by whether you want a frozen image you re-export or an always-current store the KB reads and writes.

Snapshot sink — vaelii.sqlite.snapshot

A SnapshotSink / SnapshotSource over the engine's snapshot seam (vaelii.impl.io.snapshot). It puts a KB image in a single file: the index projection today, and any of the seam's named sections as they land.

A snapshot is O(sections) bulk blob transfers, not O(records) tiny probes, so it is cheap on any store and cheapest of all on SQLite — no server to stand up, just a file. "Put my KB in a file" — for backup, for shipping a corpus to another host, for embedding a KB beside an app — is answered here, and the file is the artifact you copy.

Two properties the seam gives, and a database sharpens:

• One transaction, manifest last. The whole image writes inside a single transaction and the manifest row commits it, so a crash leaves no manifest and the source rebuilds — the file sink's "write manifest.edn last" rule, enforced by the database rather than by ordering.
• Validate or discard. The records fingerprint and index-layout version ride a column and the manifest; a mismatched image is discarded and the caller rebuilds, never trusted. The check is the engine's shared snapshot/decision.

A section written through this sink reads back frame-identical through any source — file, memory, Postgres, or SQLite.

Record store — vaelii.sqlite.record-store

A durable RecordStore (the engine's vaelii.impl.protocols/RecordStore) over a single file — a live third records backend beside core's :memory and :disk, where the disk backend is a bespoke mmap/idx pair and the snapshot is a frozen image. This one is the running store, in a file any SQLite tool can open.

The "two orders of magnitude off local" that keeps a networked records store off the query path does not apply here: embedded SQLite runs in-process, and a warm point read measured ~6µs against the disk store's ~3µs — a constant factor, not a round trip. Records are whole nippy-frozen frames keyed by handle, so a fetch thaws back type-identical; a sentex's strength rides a column read back onto the record, so mark-premise is a column update, never a frame rewrite; handles are never reissued across a reopen. One connection under WAL + synchronous=NORMAL (the disk store's durability shape), serialized on a lock, an LRU in front.

Core reaches it as the :sqlite backend, resolved lazily so the SSPL engine never loads a JDBC driver unless a KB asks for it:

(require '[vaelii.core :as v])

(def kb (v/open-kb {:backend :sqlite :dir "/path/to/kb"}))  ; records.sqlite lands there
(v/assert kb '(likes Felix Tuna) 'CxTest)
(v/close! kb)                        ; releases the connection; the file is the KB

;; a later open over the same :dir recovers the whole KB from the file alone
(def kb' (v/open-kb {:backend :sqlite :dir "/path/to/kb"}))

Using the snapshot sink

(require '[vaelii.sqlite.snapshot :as sqlite]
         '[vaelii.impl.io.snapshot :as snap])

(def ds {:dbtype "sqlite" :dbname "kb.db"})   ; a file — see the note below

;; write a KB's index projection as an image
(with-open [sink (sqlite/sqlite-sink ds "my-kb")]
  (snap/save-index! sink (:index kb) records-fingerprint))

;; read it back into an (empty) index, validated against the current records
(snap/load-index! (sqlite/sqlite-source ds "my-kb") (:index other-kb) records-fingerprint)
;; => {:index :replayed :entries n}   — or {:index :rebuild :reason …}

sqlite-sink is java.io.Closeable; commit! commits, and a close without a commit rolls the whole image back. Use with-open.

Point it at a file. The sink, the source, and ensure-schema! each open their own connection, and only a file-backed ds shares one database across them. An in-memory :memory: spec gives each connection its own database, so the schema one opens is invisible to the next — use a file, or a shared-cache datasource, if you need memory.

Development

checkouts/vaelii -> ../vaelii shadows the core dependency with dev-core source, so the seam is the checkout's, not a jar's. Run scripts/link-checkouts.sh after a fresh clone (checkouts/ is gitignored).

lein test        # always runs — SQLite needs no server, only a temp file
lein lint        # kondo + cljfmt + shellcheck + the reflection ratchet

Every test stands up a fresh temp-file database, runs, and deletes it. There is nothing to gate on and no VAELII_* variable to set — that is the difference from the Postgres sibling.

License

Apache-2.0. See LICENSE.