vaelii-postgres

Postgres targets for vaelii's storage seams — an Apache-2.0 adapter on the SSPL engine. It depends on core; core never depends on it.

The artifact is com.vaelii/postgres.

What's here

Two lanes, and they are independent — a KB can take either, both, or neither.

vaelii.postgres.record-store — the records in a database

The engine's RecordStore seam over three tables, so a KB's durable ground truth lives in a database an operator already runs. Core selects it as :pg-memory (the derived index in RAM, rebuilt on every open) or :pg-disk-log (the durable index, which is local to the host running the writer and does not travel with the KB).

What a server buys, stated narrowly:

• COPY — copy-sentexes! loads at 95.8k records/s where the per-record door manages 4.1k/s and core's :disk-log store manages 52.7k/s. The strongest single argument for this backend.
• An operator's existing everything — backup, PITR, replication, monitoring, access control, a query surface. None of it is ours to write.
• A store bigger than one disk, on a machine that is not the JVM's.

What it does not buy: a shared KB. Belief lives in the writing process's RAM, so a second process on the same database hides the first's beliefs and its retraction sweep deletes records the first still believes. That is core's single-writer contract, and a server does not weaken it by one clause — it just cannot fail a second writer fast the way a file lock does.

The round trip is what the store's shape answers to: an uncached point read is 282.6 µs against the :disk-log store's 0.26 µs warm, and 0.38 µs on a fetch-LRU hit. So the cache is the backend's viability rather than a tuning knob, the premise-strength cache is filled by the premise-ids walk that precedes every read of it, and the enumerations run on a server-side cursor because reindex and recover walk all of them.

vaelii.postgres.snapshot — the KB image in a database

A SnapshotSink / SnapshotSource over the engine's snapshot seam (vaelii.impl.io.snapshot): the index projection today, and any of the seam's named sections as they land. This is the lane with no round trip in it — an image is O(sections) bulk blob transfers, not O(records) tiny probes — so it answers "put my KB in Postgres" for backup or shipping whether or not the records live in a database at all.

Two properties the seam gives, and a database sharpens:

• One transaction, manifest last. The whole image writes inside a single transaction and the manifest row commits it, so a crash leaves no manifest and the source rebuilds — the file sink's "write manifest.edn last" rule, enforced by the database rather than by ordering.
• Validate or discard. The records fingerprint and index-layout version ride a column and the manifest; a mismatched image is discarded and the caller rebuilds, never trusted. The check is the engine's shared snapshot/decision.

A section written through this sink reads back frame-identical through any source — file, memory, or Postgres.

Using it

A KB whose records are in Postgres

(require '[vaelii.core :as v]
         '[vaelii.postgres.record-store])   ; puts the adapter on the classpath

;; the derived index in RAM, rebuilt on every open
(def kb (v/open-kb {:backend :pg-memory
                    :pg {:dbtype "postgresql" :host "localhost" :dbname "vaelii"
                         :user "vaelii" :schema "prod"}}))

;; …or the durable index, in a directory on THIS host
(def kb (v/open-kb {:backend :pg-disk-log :dir "/var/lib/vaelii/kb"
                    :pg {:jdbcUrl "jdbc:postgresql://db.internal/vaelii"}}))

(v/assert kb '(likes Muffet Tom) 'CxTest)
(v/close! kb)          ; releases the pool, and the index directory's lock

A bulk load bypasses the KB and goes straight at the store:

(require '[vaelii.postgres.record-store :as rec])

(with-open [store (rec/pg-record-store {:dbtype "postgresql" :dbname "vaelii"
                                        :schema "prod"})]
  (rec/copy-sentexes! store sentexes)          ; COPY … FROM STDIN BINARY
  (rec/copy-justifications! store deductions))

A KB image in Postgres

(require '[vaelii.postgres.snapshot :as pg]
         '[vaelii.impl.io.snapshot :as snap])

(def ds {:dbtype "postgresql" :host "localhost" :dbname "vaelii" :user "vaelii"})

;; write a KB's index projection as an image
(with-open [sink (pg/pg-sink ds "my-kb")]
  (snap/save-index! sink (:index kb) records-fingerprint))

;; read it back into an (empty) index, validated against the current records
(snap/load-index! (pg/pg-source ds "my-kb") (:index other-kb) records-fingerprint)
;; => {:index :replayed :entries n}   — or {:index :rebuild :reason …}

pg-sink is java.io.Closeable; commit! commits, and a close without a commit rolls the whole image back. Use with-open.

Development

checkouts/vaelii -> ../vaelii shadows the core dependency with dev-core source, so the seam is the checkout's, not a jar's.

lein test        # db-touching tests skip with a printed reason when no server

The record store's tests — its oracle against the in-memory reference, its durability, its COPY path — are the ones that need a server, so a change to it is landed against a run that had one:

docker compose -f docker-compose.test.yml up -d
POSTGRES_USER=vaelii POSTGRES_PASSWORD=vaelii lein test
docker compose -f docker-compose.test.yml down -v

Each test works in a schema of its own and drops it afterwards.

Point the suite at a server with VAELII_PG_URL, or the POSTGRES_HOST / POSTGRES_PORT / POSTGRES_DB / POSTGRES_USER / POSTGRES_PASSWORD variables; it defaults to localhost:5432/vaelii as the current OS user. A stray PGHOST never changes what lein test means — every database test is gated on a reachable server.

License

Apache-2.0. See LICENSE.