Liking cljdoc? Tell your friends :D

ebifriedrice.rsqlite

Reactive SQLite: register queries, receive recomputed results.

(def eng  (rsql/connect))                     ; in-memory
(def eng  (rsql/connect "file:app.db"))      ; or a file
(rsql/exec! eng "create table person (id integer primary key, name text, age int)")
(def adults (rsql/q! eng "select id, name from person where age >= ?" 18))
@adults
(rsql/exec! eng "insert into person values (1, 'Ann', 33)")
@adults                                     ; recomputed

Result atoms are plain Clojure atoms: deref reads, watch reacts, Reagent re-renders for free on both platforms. Writes go through the engine (exec!/transact!) - the posh contract; the preupdate hook feeds invalidation, recomputation runs off the writing thread, and only genuinely changed results are delivered.

The engine namespace documents the semantics (scheduling, DDL, rollback); this is the sugar layer over it, malli-specified. On the JVM the backend is the bundled org.sqlite.jni binding; on ClojureScript (later in the series) connect takes the consumer's initialized @sqlite.org/sqlite-wasm module.

Reactive SQLite: register queries, receive recomputed results.

    (def eng  (rsql/connect))                     ; in-memory
    (def eng  (rsql/connect "file:app.db"))      ; or a file
    (rsql/exec! eng "create table person (id integer primary key, name text, age int)")
    (def adults (rsql/q! eng "select id, name from person where age >= ?" 18))
    @adults
    (rsql/exec! eng "insert into person values (1, 'Ann', 33)")
    @adults                                     ; recomputed

Result atoms are plain Clojure atoms: deref reads, watch reacts,
Reagent re-renders for free on both platforms. Writes go through
the engine (exec!/transact!) - the posh contract; the preupdate
hook feeds invalidation, recomputation runs off the writing
thread, and only genuinely changed results are delivered.

The engine namespace documents the semantics (scheduling, DDL,
rollback); this is the sugar layer over it, malli-specified. On
the JVM the backend is the bundled org.sqlite.jni binding; on
ClojureScript (later in the series) connect takes the consumer's
initialized @sqlite.org/sqlite-wasm module.
raw docstring

ebifriedrice.rsqlite.backend

The seam between the pure engine and a concrete SQLite binding.

The engine (registry, invalidation, recomputation) is pure cljc logic over this protocol; every platform provides one implementation over its native binding - org.sqlite.jni on the JVM, sqlite-wasm's capi in ClojureScript. Handles are opaque: prepared statements especially are binding-specific objects the engine only ever passes back.

Method map, with the contract the engine relies on:

-exec! run one statement for effect -prepare compile one statement, reused for every recompute -run-query bind + step a prepared statement to completion and reset it for reuse; returns rows as maps -finalize! release a prepared statement -subscribe! deliver raw preupdate events (preupdate.clj's shape) and rollback notifications -column-names declared columns of a table (a zero-row select) -eqp-tables the planner's table set for a statement -autocommit? whether no transaction is open -close! close the connection; per-db hooks are cleaned up raw-connection the native handle (sqlite3 on the JVM), for statement layers layered on top - nil where meaningless (wasm)

The seam between the pure engine and a concrete SQLite binding.

The engine (registry, invalidation, recomputation) is pure cljc
logic over this protocol; every platform provides one
implementation over its native binding - org.sqlite.jni on the
JVM, sqlite-wasm's capi in ClojureScript. Handles are opaque:
prepared statements especially are binding-specific objects the
engine only ever passes back.

Method map, with the contract the engine relies on:

-exec!        run one statement for effect
-prepare      compile one statement, reused for every recompute
-run-query    bind + step a prepared statement to completion and
              reset it for reuse; returns rows as maps
-finalize!    release a prepared statement
-subscribe!   deliver raw preupdate events (preupdate.clj's shape)
              and rollback notifications
-column-names declared columns of a table (a zero-row select)
-eqp-tables   the planner's table set for a statement
-autocommit?  whether no transaction is open
-close!       close the connection; per-db hooks are cleaned up
raw-connection the native handle (sqlite3 on the JVM), for
statement layers layered on top - nil where meaningless (wasm)
raw docstring

ebifriedrice.rsqlite.backend.jvm

SqliteBackend over org.sqlite.jni: one open connection per backend, preupdate + rollback hooks installed on first subscribe.

SqliteBackend over org.sqlite.jni: one open connection per backend,
preupdate + rollback hooks installed on first subscribe.
raw docstring

ebifriedrice.rsqlite.backend.wasm

SqliteBackend over @sqlite.org/sqlite-wasm's capi + oo1 layer.

The consumer owns module initialization (one sqlite3InitModule per app - their build toolchain's import) and hands the initialized module to connect. Statements use the oo1 Stmt (prepare/bind/step/ get/reset - get converts values to JS, int64 included when it fits a double); hooks use the raw capi against db.pointer, where the _js variants of preupdate_old/new convert values for us.

JS numbers cannot carry int64 beyond 2^53; such values lose precision on this platform (the JVM keeps full longs).

SqliteBackend over @sqlite.org/sqlite-wasm's capi + oo1 layer.

The consumer owns module initialization (one sqlite3InitModule per
app - their build toolchain's import) and hands the initialized
module to connect. Statements use the oo1 Stmt (prepare/bind/step/
get/reset - get converts values to JS, int64 included when it fits
a double); hooks use the raw capi against db.pointer, where the
_js variants of preupdate_old/new convert values for us.

JS numbers cannot carry int64 beyond 2^53; such values lose
precision on this platform (the JVM keeps full longs).
raw docstring

ebifriedrice.rsqlite.capi

Thin, idiomatic Clojure over the org.sqlite.jni C-style binding.

This is the only namespace that touches the JNI classes (besides natives' loader), so upstream's experimental-API churn stays behind one seam. Everything here maps one C call to one Clojure function, converts values per column type, and turns non-OK result codes into ex-info carrying sqlite3_errmsg - the C layer never throws on its own.

Resource discipline follows the binding's golden rule: statements are finalized (with-stmt) and databases closed (close!) explicitly; the GC never frees native resources.

Thin, idiomatic Clojure over the org.sqlite.jni C-style binding.

This is the only namespace that touches the JNI classes (besides
natives' loader), so upstream's experimental-API churn stays behind
one seam. Everything here maps one C call to one Clojure function,
converts values per column type, and turns non-OK result codes
into ex-info carrying sqlite3_errmsg - the C layer never throws on
its own.

Resource discipline follows the binding's golden rule: statements
are finalized (with-stmt) and databases closed (close!)
explicitly; the GC never frees native resources.
raw docstring

ebifriedrice.rsqlite.engine

The reactive core: registry, invalidation, recomputation.

One engine wraps one backend connection. Writes go through exec! or transact!; the backend's preupdate hook materializes raw events into :pending on the writing thread (cheap swap!s only - the writer is never blocked); when the statement leaves the connection in autocommit mode a flush is scheduled off the writer. The flush resolves positional rows to column names (off-writer, as required

  • that needs a prepared statement), coalesces, matches against registered deps, re-runs affected queries and resets only those result atoms whose realized rows actually changed. sync! runs the same flush synchronously for deterministic callers (tests).

Scheduling is injectable; by default flushes go to a single background thread on the JVM and a microtask on JS, so watchers fire away from whichever thread wrote. Ordering within an engine is preserved: flushes drain until :pending is empty.

DDL (create/alter/drop through exec!) is not tracked by deps; it clears the column-name cache and re-derives every query's profile before the next recompute pass.

The registry maps id -> {:sql :params :deps :stmt :result :last}; :result is the deref-able, watchable atom returned by register!.

The reactive core: registry, invalidation, recomputation.

One engine wraps one backend connection. Writes go through exec!
or transact!; the backend's preupdate hook materializes raw events
into :pending on the writing thread (cheap swap!s only - the
writer is never blocked); when the statement leaves the connection
in autocommit mode a flush is scheduled off the writer. The flush
resolves positional rows to column names (off-writer, as required
- that needs a prepared statement), coalesces, matches against
registered deps, re-runs affected queries and resets only those
result atoms whose realized rows actually changed. sync! runs the
same flush synchronously for deterministic callers (tests).

Scheduling is injectable; by default flushes go to a single
background thread on the JVM and a microtask on JS, so watchers
fire away from whichever thread wrote. Ordering within an engine
is preserved: flushes drain until :pending is empty.

DDL (create/alter/drop through exec!) is not tracked by deps; it
clears the column-name cache and re-derives every query's profile
before the next recompute pass.

The registry maps id -> {:sql :params :deps :stmt :result :last};
:result is the deref-able, watchable atom returned by register!.
raw docstring

ebifriedrice.rsqlite.jdbc

next.jdbc on top of rsqlite's own connection.

(require '[ebifriedrice.rsqlite :as rsql] '[ebifriedrice.rsqlite.jdbc :as rjdbc] '[next.jdbc :as jdbc])

(def eng (rsql/connect)) (def ds (rjdbc/data-source eng)) (jdbc/execute! ds ["create table t (id integer primary key, s text)"]) (jdbc/execute-one! ds ["insert into t (s) values (?)" "hi"])

The data source implements javax.sql.DataSource over the engine's org.sqlite.jni handle; jdbc/execute!, execute-one!, plan/select! and jdbc/transact work, and every write lands on the same connection the preupdate hook watches - so reactive queries re-run with full column-level precision, exactly as writes through exec!/transact!.

Semantics and limits: one statement per call (the binding prepares the first statement only); setObject accepts Long, Double, String, byte [], Boolean and nil; transactions follow next.jdbc's use of setAutoCommit/commit/rollback (BEGIN on setAutoCommit(false), COMMIT/ROLLBACK on the shared handle, where the engine's rollback hook discards pending events); generated keys are not supported; close() on the connection is a no-op - the engine owns the connection's lifetime.

next.jdbc is an optional dependency: consumers add seancorfield/next.jdbc; nothing else in rsqlite needs it.

next.jdbc on top of rsqlite's own connection.

(require '[ebifriedrice.rsqlite :as rsql]
          '[ebifriedrice.rsqlite.jdbc :as rjdbc]
          '[next.jdbc :as jdbc])

(def eng (rsql/connect))
(def ds  (rjdbc/data-source eng))
(jdbc/execute! ds ["create table t (id integer primary key, s text)"])
(jdbc/execute-one! ds ["insert into t (s) values (?)" "hi"])

The data source implements javax.sql.DataSource over the engine's
org.sqlite.jni handle; jdbc/execute!, execute-one!, plan/select!
and jdbc/transact work, and every write lands on the same
connection the preupdate hook watches - so reactive queries
re-run with full column-level precision, exactly as writes
through exec!/transact!.

Semantics and limits: one statement per call (the binding
prepares the first statement only); setObject accepts Long,
Double, String, byte [], Boolean and nil; transactions follow
next.jdbc's use of setAutoCommit/commit/rollback (BEGIN on
setAutoCommit(false), COMMIT/ROLLBACK on the shared handle, where
the engine's rollback hook discards pending events); generated
keys are not supported; close() on the connection is a no-op -
the engine owns the connection's lifetime.

next.jdbc is an optional dependency: consumers add
seancorfield/next.jdbc; nothing else in rsqlite needs it.
raw docstring

ebifriedrice.rsqlite.match

Pure invalidation decisions: which registered queries must re-run given the events a transaction produced.

This is the posh datom-matcher's analogue and the cheap check the engine runs before re-executing anything. Events arrive per row with per-row changed columns (already name-resolved); coalesce merges them per table; affected? applies the rules:

  • :all dependencies always re-run;
  • a table not in the query's tables never re-runs it;
  • inserted or deleted rows re-run every query on the table (a new row appears in results regardless of which columns it set);
  • a coarse event (unknown columns) re-runs everything on the table;
  • a wildcard over the table re-runs on any change to it;
  • otherwise when a changed column is among the columns bound to that table (alias-qualified or single-table references), or among the query's scope-unbound columns (bare references in multi-table scopes).

All names must already be normalised (sqldeps/norm-name).

Pure invalidation decisions: which registered queries must re-run
given the events a transaction produced.

This is the posh datom-matcher's analogue and the cheap check the
engine runs before re-executing anything. Events arrive per row
with per-row changed columns (already name-resolved); coalesce
merges them per table; affected? applies the rules:

- :all dependencies always re-run;
- a table not in the query's tables never re-runs it;
- inserted or deleted rows re-run every query on the table (a new
  row appears in results regardless of which columns it set);
- a coarse event (unknown columns) re-runs everything on the table;
- a wildcard over the table re-runs on any change to it;
- otherwise when a changed column is among the columns bound to
  that table (alias-qualified or single-table references), or
  among the query's scope-unbound columns (bare references in
  multi-table scopes).

All names must already be normalised (sqldeps/norm-name).
raw docstring

ebifriedrice.rsqlite.natives

Load the SQLite JNI native library.

Neither of the obvious mechanisms works on a modern JVM:

  • System/loadLibrary searches java.library.path, which the JVM snapshots at startup - a jar-bundled library is invisible to it;
  • System/load called from our (dynamically compiled) Clojure code registers the library with the calling class's classloader, and CApi - a jar class - cannot bind natives from it.

So the jar ships a one-method Java shim (built by script/native-setup.sh): ensure-loaded! extracts the bundled native for this platform into a stable, content-addressed cache directory and asks the shim - a class in CApi's own classloader - to System.load it. CApi's static block is patched to a no-op install() call, so nothing else tries to load.

The cache directory (XDG_CACHE_HOME/rsqlite-natives or LOCALAPPDATA\rsqlite-natives on windows) is keyed by the library's sha-256: extraction and the platform's fresh-file-virus-scan happen once per library version, not once per JVM launch.

Without a bundled resource, the shim falls back to System.loadLibrary, honouring whatever java.library.path the host configured at launch.

Load the SQLite JNI native library.

Neither of the obvious mechanisms works on a modern JVM:

- System/loadLibrary searches java.library.path, which the JVM
  snapshots at startup - a jar-bundled library is invisible to it;
- System/load called from our (dynamically compiled) Clojure code
  registers the library with the calling class's classloader, and
  CApi - a jar class - cannot bind natives from it.

So the jar ships a one-method Java shim (built by
script/native-setup.sh): ensure-loaded! extracts the bundled
native for this platform into a stable, content-addressed cache
directory and asks the shim - a class in CApi's own classloader -
to System.load it. CApi's static block is patched to a no-op
install() call, so nothing else tries to load.

The cache directory (XDG_CACHE_HOME/rsqlite-natives or
LOCALAPPDATA\rsqlite-natives on windows) is keyed by the
library's sha-256: extraction and the platform's
fresh-file-virus-scan happen once per library version, not once
per JVM launch.

Without a bundled resource, the shim falls back to
System.loadLibrary, honouring whatever java.library.path the host
configured at launch.
raw docstring

ebifriedrice.rsqlite.preupdate

Materialize preupdate-hook callbacks as data.

Inside the callback everything must be copied out immediately - sqlite3_value handles die when it returns - and nothing else may touch the connection (the C rules forbid it). So the callback does the minimum: read column count and the old/new row through preupdate accessors, convert values, hand one map per row to on-event, return. Column names are resolved later, off the writer, by the engine, because asking sqlite for them requires preparing a statement.

Event shape (positional, names pending):

{:op :insert|:update|:delete
 :table "t"                ; as sqlite reports it
 :rowid-old n? :rowid-new n? ; nil-able; undefined for
                              ; WITHOUT ROWID tables
 :old [v*]? :new [v*]?}      ; old absent for insert,
                              ; new absent for delete
Materialize preupdate-hook callbacks as data.

Inside the callback everything must be copied out immediately -
sqlite3_value handles die when it returns - and nothing else may
touch the connection (the C rules forbid it). So the callback does
the minimum: read column count and the old/new row through
preupdate accessors, convert values, hand one map per row to
on-event, return. Column names are resolved later, off the writer,
by the engine, because asking sqlite for them requires preparing a
statement.

Event shape (positional, names pending):

    {:op :insert|:update|:delete
     :table "t"                ; as sqlite reports it
     :rowid-old n? :rowid-new n? ; nil-able; undefined for
                                  ; WITHOUT ROWID tables
     :old [v*]? :new [v*]?}      ; old absent for insert,
                                  ; new absent for delete
raw docstring

ebifriedrice.rsqlite.sql

SQL SELECT parsing, purpose-fit for dependency tracking.

The grammar mirrors the SELECT-related productions of SQLite's own parse.y but is deliberately loose wherever looseness cannot hide a dependency: expression interiors are flat term/op token sequences, so operator precedence, CASE/CAST structure and window frames never matter - only that every identifier occurrence is captured as a ColumnRef, a table reference, or an alias.

Two soundness rules the looseness depends on:

  • No opaque blobs. Structural keywords (FROM, WHERE, JOIN, ...) are never consumable inside expressions, so a clause skeleton cannot be swallowed by an expression and lose a table reference. Bare IN is only tokenisable when a parenthesis follows, so the pathological x IN some_table fails to parse and falls back conservatively instead of silently losing a table.

  • A leading '*' cannot start an expression (terms first, operators after), so SELECT * is never misparsed as multiplication and a star selector is never lost.

Remaining ambiguity is over-approximation, never loss: a function name parsed as a column, a window name parsed as a column - both merely over-trigger recomputation.

Anything unparseable is a parse failure; sqldeps maps that to the conservative :all dependency set. Identifier case is untouched here; sqldeps normalises, since SQLite's ASCII case-insensitivity applies equally to names coming from the database.

SQL SELECT parsing, purpose-fit for dependency tracking.

The grammar mirrors the SELECT-related productions of SQLite's own
parse.y but is deliberately loose wherever looseness cannot hide a
dependency: expression interiors are flat term/op token sequences,
so operator precedence, CASE/CAST structure and window frames never
matter - only that every identifier occurrence is captured as a
ColumnRef, a table reference, or an alias.

Two soundness rules the looseness depends on:

- No opaque blobs. Structural keywords (FROM, WHERE, JOIN, ...) are
  never consumable inside expressions, so a clause skeleton cannot
  be swallowed by an expression and lose a table reference. Bare
  IN is only tokenisable when a parenthesis follows, so the
  pathological `x IN some_table` fails to parse and falls back
  conservatively instead of silently losing a table.

- A leading '*' cannot start an expression (terms first, operators
  after), so `SELECT *` is never misparsed as multiplication and a
  star selector is never lost.

Remaining ambiguity is over-approximation, never loss: a function
name parsed as a column, a window name parsed as a column - both
merely over-trigger recomputation.

Anything unparseable is a parse failure; sqldeps maps that to the
conservative :all dependency set. Identifier case is untouched
here; sqldeps normalises, since SQLite's ASCII case-insensitivity
applies equally to names coming from the database.
raw docstring

ebifriedrice.rsqlite.sqldeps

Derive a query's dependency profile from a parse tree.

The profile is what the matcher compares update events against:

{:tables          #{table-name}
 :table-columns   {table-name #{column-name}} ; alias-bound
 :columns         #{column-name}              ; scope-unbound
 :wildcard-tables #{table-name}}              ; any change re-runs

Extraction is scoped, walking one Select level at a time with the CTE names visible at that level, so CTE shadowing a real table name in a sibling scope cannot subtract that real table. Columns bind to their table when the reference is alias/table-qualified (u.age, users.age) or when the scope reads a single table; bare references in multi-table scopes stay unbound and match any queried table - over-triggering on shared names there only, never missing. Stars resolve through each core's FROM aliases, including CTE names (to the CTE body's tables, recursively) and FROM subqueries. Anything the grammar cannot parse returns nil here; deps blends with EQP and degrades to table granularity.

All names are normalised - quotes stripped, ASCII-folded - because SQLite treats identifiers as ASCII case-insensitive and the database reports column names in their declared case.

Derive a query's dependency profile from a parse tree.

The profile is what the matcher compares update events against:

    {:tables          #{table-name}
     :table-columns   {table-name #{column-name}} ; alias-bound
     :columns         #{column-name}              ; scope-unbound
     :wildcard-tables #{table-name}}              ; any change re-runs

Extraction is scoped, walking one Select level at a time with the
CTE names visible at that level, so CTE shadowing a real table
name in a sibling scope cannot subtract that real table. Columns
bind to their table when the reference is alias/table-qualified
(u.age, users.age) or when the scope reads a single table; bare
references in multi-table scopes stay unbound and match any
queried table - over-triggering on shared names there only,
never missing. Stars resolve through each core's FROM aliases,
including CTE names (to the CTE body's tables, recursively) and
FROM subqueries. Anything the grammar cannot parse returns nil
here; `deps` blends with EQP and degrades to table granularity.

All names are normalised - quotes stripped, ASCII-folded - because
SQLite treats identifiers as ASCII case-insensitive and the
database reports column names in their declared case.
raw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
←Move to previous article
→Move to next article
Ctrl+/Jump to the search field
× close