vaelii.host.serve — com.vaelii/vaelii 0.30.0

Headless EDN-over-HTTP daemon: one JVM owns one KB and serves it to remote clients (vaelii.host.client). A thin reitit-ring + jetty layer over vaelii.core, the network dual of the in-process API.

Wire format is EDN. A sentence is a symbol s-expression — (dog Muffet), ?x, (genl dog animal) — which EDN round-trips losslessly; JSON would mangle the symbols. The body of every call is {:op <keyword> :args [...]}, and the reply is {:ok true :result …} or {:ok false :error "…"}. EDN is read with clojure.edn/read-string (never clojure.core/read-string), so an untrusted body cannot evaluate code — EDN has no reader-eval.

A refusal's :type is a plain keyword — :body-too-large, :not-edn, :cross-origin, :bad-host — and so is the :type an engine ex-info carries through. The protocol is what a client written against another build discriminates on, so it cannot be qualified by the namespace that happens to serve it: a ::-qualified keyword names this namespace, and a client matching on it would be matching on where the daemon's code lives.

The daemon is the single writer (docs/storage.md, the single-writer contract): it owns the one process allowed to mutate the store, so it serializes every op through one monitor. Concurrent client writes therefore apply one at a time and cannot interleave; reads pay the same lock, which is conservative but keeps the contract simple.

Only the allowlisted ops are reachable (ops). Each is a vaelii.core fn with the KB supplied by the daemon — the client sends only the op and the remaining args — so no client can reach an arbitrary var. Sentex records in a result are projected to plain maps before they hit the wire (the sentex-map contract), so the client reads them back without the impl record class.

The change feed is the one thing that is not a vaelii.core fn (feed-ops), and it is a table of its own for that reason: core/watch takes a callback, so what a remote caller holds open instead is a subscription with a cursor — :watch, :poll, :unwatch, :watchers, over the per-handler registry app builds (vaelii.host.subscribe, docs/feed.md). A :poll that waits runs outside the monitor; everything else about them is an ordinary EDN op.

One shared bearer token authenticates the caller. With VAELII_API_TOKEN set (guard/api-token), every request presents Authorization: Bearer <token> or is answered 401 with a WWW-Authenticate: Bearer challenge; GET /health is the one route that answers without it. One token for the process, not a session and not an identity — per-caller identity is a reverse proxy's job, and this is the check that has to exist below it. Binding anything but loopback requires a token (-main refuses to start otherwise); on the loopback default it is optional, and a daemon without one is drivable by every process on the machine.

A per-principal token table replaces the shared token when VAELII_TOKEN_TABLE names one (guard/token-table): each token carries a grant naming a principal and the contexts it writes, and a write op lands only there, recorded under that principal (bound-write). Reads stay open to every token in the table.

vaelii.host.guard covers what a token does not, and matters most on the open loopback daemon: POST /op requires Content-Type: application/edn, refuses a cross-origin Origin, and answers only to a Host naming the interface it was started on. Together those stop a page the operator happens to visit from driving the KB over loopback — which binding to loopback alone does not.

*served-kbs* clj

nil, or a fn from a string a request sends to the KB this server serves under that name, or nil when it serves none. The browser binds it to its catalog around handle-op (vaelii.browser.web/op-post); the daemon serves one KB and leaves it nil.

-main clj

(-main & args)

Run the daemon in the foreground. Args: [port [dir]] [--listen ADDR] [--starter], in any order — dir opens the store there under the backend its files were written by, or a new durable :disk-log store when it holds none (dir-backend); either is recovered on open, so it persists across restarts. With no dir the KB is in-memory and lives only as long as the process.

--starter loads the shipped starter schema (vaelii.host.starter/load-into) into the KB after it opens and before the port is bound, so GET /health answers only once the schema is in. The load asserts every starter sentence: into a directory that already holds them, each dedups to its stored handle, and the load still reads every file.

lein run -m vaelii.host.serve 4200 /var/lib/vaelii lein run -m vaelii.host.serve 4200 --starter ; in memory, with the schema lein run -m vaelii.host.serve 4200 /var/lib/vaelii --listen 0.0.0.0 ; opt-in

It binds loopback unless --listen says otherwise, for the reason on loopback above: POST /op writes, and it is the KB's only writer. What it binds and what it requires are one decision (auth-posture), so they are stated together:

• --listen names a non-loopback address ⇒ VAELII_API_TOKEN is required. Without one it is a line on stderr and exit 2, a code of its own so a supervisor tells a missing credential from the configuration typos below.
• Loopback — the default, and --listen 127.0.0.1 said out loud — ⇒ the token is used when set, and its absence is a startup warning naming the flag that would require one.

Naming an address also drops the Host allowlist (guard/allowed-hosts), since the name you reach it by is then yours to know; set VAELII_ALLOWED_HOSTS to keep the check. Left unset, the daemon starts anyway — a reverse proxy setting its own Host needs exactly this — and host-posture turns the gap into a startup warning rather than a silence. A --listen with no address, an unknown flag, or a stray argument is one line and exit 1, like the port typo below.

app clj

(app kb)

(app kb opts)

The ring handler for a KB — pure request -> response, so it is tested without a socket. One monitor per handler serializes the ops (the single-writer contract).

:host names the interface this handler will be served on, which fixes the Host values it answers to (guard/allowed-hosts). On the loopback default that refuses a rebound DNS name, the one attack same-origin? cannot see.

:token is the shared bearer token every request must present. Omitted, it is VAELII_API_TOKEN (guard/api-token), so a daemon and a client on one host agree without either being configured; an explicit nil serves open, which is what a test of the other refusals needs — a handler that 401s first exercises none of them.

:token-table is the per-principal token table, a map or the path of an EDN file (guard/check-token-table). Omitted with :token omitted too, it is VAELII_TOKEN_TABLE's (guard/token-table); omitted beside a :token, or an explicit nil, it is none. Set, it replaces :token: a request presents a token the table holds, and a write op lands only in the contexts that token's grant names (bound-write). A table that does not check is refused (:bad-token-table).

feed-ops clj

The change-feed operations, keyed by op keyword — (fn [ctx args]) over a ctx of {:kb :registry :monitor}, plus an optional :admit: handle-op's, which :watch calls inside the monitor before it registers anything.

Each says its own relationship to the daemon's write monitor, which is the one thing about them that is not like an engine op. :watch and :unwatch take it: they are instantaneous, and taking it makes the boundary exact — every settle that finished before a :watch returned is outside the subscription's feed, and every one that starts after it is inside. :poll must not, because a long poll parks: inside the monitor it would block every writer for the duration of its wait, turning a feature about liveness into a global stall. :watchers does not either — it reads the registry and expires what has expired, and a listing that had to queue behind a bulk load is a listing an operator asks for while the daemon is busy.

handle-op clj

(handle-op kb registry monitor req)

(handle-op kb registry monitor req admit)

(handle-op kb registry monitor req admit host)

Run one {:op :args} request under the write lock and answer with EDN. The op runs on a thread of its own (off-worker), so stopping the server interrupts no KB write.

Public because the browser serves the same protocol: vaelii.browser.web answers POST /op by calling this with its active KB, its own subscription registry and its own write monitor, so a native client reads the browser's KB with the daemon's op table, guards and refusals (docs/operations.md, "The browser serves the protocol too").

The two guards run before the body is read. POST /op is the write route of the single writer, and on an open loopback daemon nothing above has identified the caller, so a page the operator merely visits must not be able to drive it: guard/edn-body? forces a CORS preflight this daemon cannot answer, and guard/same-origin? refuses a browser that sent someone else's origin. See vaelii.host.guard.

Two tables are looked up, in order: the vaelii.core allowlist (ops, run under the monitor), then the daemon's own change-feed ops (feed-ops, which take the handler's subscription registry and decide about the monitor themselves — a long poll parks, and a parked poll holding it would block every writer).

A write op (write-ops) is held to the grant the request's token carries (guard/wrap-token-table puts it on the request), inside the monitor, by bound-write. A request with no grant, the browser's included, writes as before, except that it may not name the provenance :principal.

admit is nil, or a fn of no arguments the op runs behind: called inside the monitor, before an engine op and before :watch registers, it answers nil to let the op run or {:status :type :error} to refuse it with that reply. The browser passes one because its KB can be released while a request waits for the monitor, and only a check made inside the monitor sees that release.

host is nil, or {:ops :refuse} from a process that serves this protocol beside something else: :ops maps op keywords to fns of the op's args the host answers itself, looked up before both tables and run outside the monitor, since they read the process rather than the KB; :refuse is a fn of no arguments asked before any other known op, nil to let it run or {:status :ok false :type :error} to refuse it. The browser answers its catalog and job registry through the first, and refuses a KB op while a job writes the active KB through the second — which leaves the catalog readable while the KB it lists loads.

http-threads clj

How many worker threads the daemon's HTTP server runs.

Stated rather than defaulted, because it is one half of a pair: a parked long poll holds one of these for the length of its wait, so subscribe/max-parked has to stay well under it or the feature that exists for liveness becomes the thing that stalls the daemon. Left implicit the two numbers were 50 and 64, the wrong way round, and nothing said so — 55 parked polls took /health from 62 ms to 26 s. serve_test asserts the relationship.

kbless-ops clj

The ops whose vaelii.core fn takes no KB (op* above). Held as data rather than left implicit in the closures, because a generator reading this table has to know whether an op's first vaelii.core parameter is the KB the daemon supplies or an argument the caller sends: vaelii.host.client's wrappers (arity for arity) read it here, and so does any caller that builds a call schema from ops.

op-names clj

Every op keyword this daemon answers, sorted — the vaelii.core allowlist and the daemon's own together. What an :unknown-op refusal hands back, so a caller discovering the surface sees one list rather than the larger half of two.

ops clj

The reachable operations, keyed by op keyword. Reads, writes, and introspection — the working set a remote caller needs. A vaelii.core fn that takes a KB is either a row here or a row of docs/operations.md's table of fns that are not ops, and the rule that page states decides which: data in and out, the KB rather than the process, a write through the checks every served write runs, and a cost a request can bound or one that grows with the KB. serve_parity_test fails on a fn that is in neither.

Every entry is wrapped by bounded, which is a no-op for an op search-bounds does not name, and by write-bounded, a no-op for an op write-bounds does not name.

port clj

(port server)

The actual TCP port a started Server is listening on — the ephemeral one when it was started with :port 0, read off its first connector.

search-bounds clj

Which of the two ceilings apply to which op, and under which key the op reads it.

A read that expands rules runs on the daemon's single write monitor, so its cost is not the caller's alone: every other request queues behind it. The clock ceiling, :max-ms (config/max-query-ms), bounds that cost at every entry point that takes a clock, and such a row names no depth: a depth ceiling there would refuse a complete answer the clock already bounds. :query, :query?, :query-status and :argue take no clock (query-opt-keys has no :max-ms), so the depth ceiling, :max-depth (config/max-query-depth), is their bound. An anytime budget map is :max-ms and takes no default of its own.

The table holds the search bounds of reads, and no write's bound. A read with no dial of its own is not on it: :sentexes-matching has nothing to raise, so there is nothing to clamp. The four backward-search entry points do have one (core/ask-opt-keys, core/prove-opt-keys), and are held to the clock ceiling.

Eight writes name a bound this table does not hold. :assert, :try-assert, :assert-many, :assert-rule and :forward-chain read :max-depth, :max-term-depth and :max-derivations (core/assert-opt-keys, forward-chain's accepted keys), and :edit, :edit-with-consequences and :preview read the same three keys off each batch entry's opts. All three bound the chaining fixpoint (chain/default-chain-opts: a term-nesting bound of 8 and no other when the request names none), which runs on the same monitor for its whole length, and a request may name any of them at any size. Every one but :forward-chain and :preview also reads :max-ms, which write-bounds holds to the write ceiling. :forward-chain reads :max-ms and no ceiling fills it in; :preview takes no clock. docs/operations.md states the same beside the ceilings.

start clj

(start kb {:keys [port host] :or {port 4200 host loopback} :as opts})

Start the daemon over kb and return the running jetty Server (:join? false, so the caller controls its lifetime — a test stops it in a finally). :port 0 binds an ephemeral port; read the actual one with port.

:host defaults to loopback; pass an address ("0.0.0.0") to bind publicly, and read the note on loopback before doing so. :token and :token-table are app's, each forwarded only when the key is there, so an omitted one still reads its variable and an explicit nil :token serves open on loopback. An address with neither a token nor a table is refused (guard/require-token!), and so is a table that does not check (:bad-token-table), both before anything binds, as -main refuses them.

.stop on the server first runs stop-ops: an op that has not entered the monitor answers 503 :stopping, and the stop returns after the op inside it finishes.

write-bounds clj

The chaining writes the daemon holds to config/max-write-ms, and where each one's option map sits: :option-arity is the argument count that ends in the map, and :fill the vaelii.core defaults between the last argument a caller may send and the map.

A chaining write runs on the daemon's single write monitor, so an operator may hold it to a clock as the reads search-bounds names are: a request may name a smaller :max-ms and is refused (:over-ceiling) for a larger one, and an absent one is the ceiling. A write the clock ends is rolled back (docs/inference.md, "Stopping a run").

write-ops clj

The ops that store, remove or annotate a sentex a request names, and :export, which writes the whole KB to the server's host: a token-table grant bounds them (bound-write). The other ops read, store nothing (:check, :check-edit, :preview), or store what the request does not name: :forward-chain what the rules derive, :abduce its hypotheses in a scratch context (docs/operations.md, "The token table").