Column codecs every persistence backend shares: entity ids and references, keyword, timestamp and JSON TEXT columns, and turn status values. A leaf namespace, so the facade, the SQLite backend and the public API require it without a load cycle.
Column codecs every persistence backend shares: entity ids and references, keyword, timestamp and JSON TEXT columns, and turn status values. A leaf namespace, so the facade, the SQLite backend and the public API require it without a load cycle.
Persistence facade: the Store protocol, the connection lifecycle and the
guarantees every backend gets before it sees a call.
SQLite is the one backend Vis ships. Its namespace loads on the first store
operation, not with this facade (see sqlite), so commands that never touch
the store skip ~480 ms of JDBC/Hikari/Flyway class loading on a cold JVM.
Every Store op forwards to that backend, which hands the facade its
implementation through store-implementation: a backend missing an op fails
to compile.
Frontends still call db-error->user-message here; the backend owns the
actual translation. Same for the store-staleness check the process-wide
shared connection uses.
Persistence facade: the `Store` protocol, the connection lifecycle and the guarantees every backend gets before it sees a call. SQLite is the one backend Vis ships. Its namespace loads on the first store operation, not with this facade (see `sqlite`), so commands that never touch the store skip ~480 ms of JDBC/Hikari/Flyway class loading on a cold JVM. Every `Store` op forwards to that backend, which hands the facade its implementation through `store-implementation`: a backend missing an op fails to compile. Frontends still call `db-error->user-message` here; the backend owns the actual translation. Same for the store-staleness check the process-wide shared connection uses.
SQLite store - V1 schema implementation.
Implements the persistence facade's Store protocol: backend, at the end
of this file, hands the facade every op through store-implementation,
which clj-kondo cannot expand, so the ns-level config above silences
:unused-public-var for the whole file. The call surface is verified
through the storage facade tests.
Tables (V1__schema.sql): session_soul, session_state, session_turn_soul, session_turn_state, session_turn_iteration, llm_routing_event, extension_aggregate, log
Connection lifecycle: (db-open! db-spec) -> {:datasource ds :path ...} (db-close! store) -> idempotent dispose
SQLite store - V1 schema implementation.
Implements the persistence facade's `Store` protocol: `backend`, at the end
of this file, hands the facade every op through `store-implementation`,
which clj-kondo cannot expand, so the ns-level config above silences
`:unused-public-var` for the whole file. The call surface is verified
through the storage facade tests.
Tables (V1__schema.sql):
session_soul, session_state,
session_turn_soul, session_turn_state,
session_turn_iteration, llm_routing_event,
extension_aggregate,
log
Connection lifecycle:
(db-open! db-spec) -> {:datasource ds :path ...}
(db-close! store) -> idempotent disposeSpace reclamation for a file-backed SQLite store.
SQLite never hands the file back on its own: auto_vacuum is off, so pages a
delete or a DROP COLUMN frees go on the FREELIST and are only reused by
later writes. That is the right default — reuse costs nothing — but a one-off
bulk reclaim (retiring a column across a whole transcript) leaves hundreds of
megabytes the file keeps until someone runs VACUUM.
maybe-vacuum! is that someone, on the same fortnight window as
foundation.housekeeping retention: at most once per vacuum-interval-days
per store, and only when the freelist is worth the rewrite — at least
vacuum-min-free-bytes AND vacuum-min-free-fraction of the file. A compact
store is never rewritten, so the usual answer is three PRAGMAs and no I/O.
VACUUM takes SQLite's exclusive lock and rewrites the whole file (measured:
15 s and 2.4 GB -> 2.0 GB on a real store), so it runs OFF the open path —
vacuum-async! on a lowest-priority daemon thread after a settling delay,
never inside a transaction. Readers and writers in this process or another
one block for its duration and then continue, which the 30 s busy timeout and
the write-retry ladder in core absorb. A process that exits first, or a
db-close! that aborts the lease mid-rewrite, simply leaves the store due at
the next start: the rewrite is transactional, so an interrupted VACUUM rolls
back rather than damaging anything.
The vis.db.vacuum marker beside the store is BOTH the clock (its mtime is
the last successful vacuum) and the cross-process mutex (an exclusive
FileLock held for the rewrite). It is created only when a vacuum is
actually attempted and deleted again when one fails, so a store that never
needed reclaiming carries no marker and stays due.
Space reclamation for a file-backed SQLite store. SQLite never hands the file back on its own: `auto_vacuum` is off, so pages a delete or a `DROP COLUMN` frees go on the FREELIST and are only reused by later writes. That is the right default — reuse costs nothing — but a one-off bulk reclaim (retiring a column across a whole transcript) leaves hundreds of megabytes the file keeps until someone runs `VACUUM`. `maybe-vacuum!` is that someone, on the same fortnight window as `foundation.housekeeping` retention: at most once per `vacuum-interval-days` per store, and only when the freelist is worth the rewrite — at least `vacuum-min-free-bytes` AND `vacuum-min-free-fraction` of the file. A compact store is never rewritten, so the usual answer is three PRAGMAs and no I/O. VACUUM takes SQLite's exclusive lock and rewrites the whole file (measured: 15 s and 2.4 GB -> 2.0 GB on a real store), so it runs OFF the open path — `vacuum-async!` on a lowest-priority daemon thread after a settling delay, never inside a transaction. Readers and writers in this process or another one block for its duration and then continue, which the 30 s busy timeout and the write-retry ladder in `core` absorb. A process that exits first, or a `db-close!` that aborts the lease mid-rewrite, simply leaves the store due at the next start: the rewrite is transactional, so an interrupted VACUUM rolls back rather than damaging anything. The `vis.db.vacuum` marker beside the store is BOTH the clock (its mtime is the last successful vacuum) and the cross-process mutex (an exclusive `FileLock` held for the rewrite). It is created only when a vacuum is actually attempted and deleted again when one fails, so a store that never needed reclaiming carries no marker and stays due.
Flyway-backed schema migration runner.
Lives in the SQLite extension because:
flyway-database-nc-sqlite)
is required to recognize jdbc:sqlite: URLs - already
declared in this extension's deps.edn.flyway-core; making it a per-backend dep keeps the root
package free of the migration toolchain.migrate! from
com.blockether.vis.sdk, but it had exactly one caller
Public API:
(migrate! datasource locations) - apply every Flyway
migration found at the given classpath locations to the
supplied DataSource. Returns the datasource for thread-style
chaining.
:baseline-on-migrate true so existing databases without a
flyway_schema_history table get one on first run. :mixed true
so SQL files with mixed transactional + DDL statements work under
SQLite.
GraalVM native-image note: Flyway discovers migrations by LISTING the
classpath location directory, which native-image does not support (it can
getResource a specific file but not enumerate a dir). So build.clj writes
an _index.edn of filenames next to each migration dir, and here we feed
Flyway an explicit ResourceProvider built from those exact paths, paired
with an empty Java migration ClassProvider for our SQL-only schema. Both
providers are required to bypass scanning. On the JVM (no index) we fall
back to Flyway's normal location scanning.
Flyway-backed schema migration runner.
Lives in the SQLite extension because:
1. The dialect-specific Flyway driver (`flyway-database-nc-sqlite`)
is required to recognize `jdbc:sqlite:` URLs - already
declared in this extension's deps.edn.
2. Flyway is the only backend-side concern using
`flyway-core`; making it a per-backend dep keeps the root
package free of the migration toolchain.
3. The previous arrangement shipped a generic `migrate!` from
`com.blockether.vis.sdk`, but it had exactly one caller
- this extension. Other backends will ship their own
migration entry point in their own jar.
Public API:
`(migrate! datasource locations)` - apply every Flyway
migration found at the given classpath `locations` to the
supplied `DataSource`. Returns the datasource for thread-style
chaining.
`:baseline-on-migrate true` so existing databases without a
`flyway_schema_history` table get one on first run. `:mixed true`
so SQL files with mixed transactional + DDL statements work under
SQLite.
GraalVM native-image note: Flyway discovers migrations by LISTING the
classpath location directory, which native-image does not support (it can
`getResource` a specific file but not enumerate a dir). So `build.clj` writes
an `_index.edn` of filenames next to each migration dir, and here we feed
Flyway an explicit `ResourceProvider` built from those exact paths, paired
with an empty Java migration `ClassProvider` for our SQL-only schema. Both
providers are required to bypass scanning. On the JVM (no index) we fall
back to Flyway's normal location scanning.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |