A command-line driver for a KB — the shell dual of the in-process API, launched with
lein run -m vaelii.host.cli <cmd> <args…>. It runs the engine in-process (no
daemon); to talk to a running daemon use vaelii.host.client instead.
lein run -m vaelii.host.cli assert '(dog Muffet)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli query '(dog ?x)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli why 3 --dir /tmp/kb lein run -m vaelii.host.cli export /tmp/dump --dir /tmp/kb lein run -m vaelii.host.cli repl --starter # interactive, starter schema lein cli help # every command and what it takes
help is a word rather than only a flag because Leiningen answers lein cli --help
itself, printing the alias expansion — the flag never reaches this namespace through
the alias, though it does through the full lein run -m vaelii.host.cli --help.
Backend. --dir <path> opens the store there under the backend its files were
written by (v/store-backend), or a new durable :disk-log store when it holds none —
recovered on open, so a fact asserted in one invocation is there in the next. A --dir
whose parent does not exist is refused (open-kb-from) rather than created.
upgrade opens a store, brings its reasoning image and index image up to this build, and
closes it (upgrade!); --verify recovers anyway and compares the two images.
With no --dir the KB is
in-memory and lives only for the process — useful for repl or a single compound
session, pointless across one-shot commands. --starter loads the shipped schema
(types, contexts, relation rules) so you can explore the ontology. --strength monotonic marks an assert or assert-rule known-true. export takes --variant records|records+index and --compression gzip|xz|none.
A flag belongs to the commands that read it (command-flags), and one carried by
a command that does not is refused rather than dropped — those three are the driver's
and go anywhere, the rest do not. A value it cannot honour is refused on the same
argument: --format texr and --depth twice name nothing.
stdout is the answer. err! keeps a refusal off it, and on-stderr keeps the
engine's own log lines off it too — Trove's console backend prints to *out*, which
here is what a script redirects. A refusal is one stderr line, error: [<:type>] <message> (refusal-line), so a script branches on the keyword and not on the prose.
One writer. A --dir KB takes the single-writer file lock (docs/storage.md), so
the CLI and a daemon cannot own the same directory at once — by design. diff and
upgrade open no KB of the run's (without-a-kb), so diff answers beside a daemon
that holds --dir.
A command-line driver for a KB — the shell dual of the in-process API, launched with `lein run -m vaelii.host.cli <cmd> <args…>`. It runs the engine in-process (no daemon); to talk to a running daemon use `vaelii.host.client` instead. lein run -m vaelii.host.cli assert '(dog Muffet)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli query '(dog ?x)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli why 3 --dir /tmp/kb lein run -m vaelii.host.cli export /tmp/dump --dir /tmp/kb lein run -m vaelii.host.cli repl --starter # interactive, starter schema lein cli help # every command and what it takes `help` is a word rather than only a flag because Leiningen answers `lein cli --help` itself, printing the alias expansion — the flag never reaches this namespace through the alias, though it does through the full `lein run -m vaelii.host.cli --help`. **Backend.** `--dir <path>` opens the store there under the backend its files were written by (`v/store-backend`), or a new durable `:disk-log` store when it holds none — recovered on open, so a fact asserted in one invocation is there in the next. A `--dir` whose parent does not exist is refused (`open-kb-from`) rather than created. `upgrade` opens a store, brings its reasoning image and index image up to this build, and closes it (`upgrade!`); `--verify` recovers anyway and compares the two images. With no `--dir` the KB is in-memory and lives only for the process — useful for `repl` or a single compound session, pointless across one-shot commands. `--starter` loads the shipped schema (types, contexts, relation rules) so you can explore the ontology. `--strength monotonic` marks an `assert` or `assert-rule` known-true. `export` takes `--variant records|records+index` and `--compression gzip|xz|none`. **A flag belongs to the commands that read it** (`command-flags`), and one carried by a command that does not is refused rather than dropped — those three are the driver's and go anywhere, the rest do not. A *value* it cannot honour is refused on the same argument: `--format texr` and `--depth twice` name nothing. **stdout is the answer.** `err!` keeps a refusal off it, and `on-stderr` keeps the engine's own log lines off it too — Trove's console backend prints to `*out*`, which here is what a script redirects. A refusal is one stderr line, `error: [<:type>] <message>` (`refusal-line`), so a script branches on the keyword and not on the prose. **One writer.** A `--dir` KB takes the single-writer file lock (docs/storage.md), so the CLI and a daemon cannot own the same directory at once — by design. `diff` and `upgrade` open no KB of the run's (`without-a-kb`), so `diff` answers beside a daemon that holds `--dir`.
A thin EDN-over-HTTP client for the vaelii daemon (vaelii.host.serve). Runs no
engine: it POSTs {:op :args} and reads the result back, over JDK java.net.http
(no dependency — JDK 21 ships it).
Every call threads an explicit connection handle as its first argument —
(query conn '(dog ?x) 'Ctx) — the network mirror of vaelii.core's explicit-kb
API. A conn from client holds a reusable HttpClient; no socket opens until a
call. A daemon reply of {:ok false} becomes an ex-info carrying the daemon's
:error, :type and :status, so a remote naming/disjointness refusal surfaces like
a local one and the coarse client-fault/server-fault split the status carries is
readable without writing the request by hand.
The bearer token rides on the request the daemon requires it on: the conn
carries it (VAELII_API_TOKEN unless :token says otherwise) and every call sets one
more header on the builder it was already using. No dependency, no client state, and
the conn is still a map you can read.
One wrapper per op, and they are generated (vaelii.regen-client, lein regen-client). The daemon's op table is the single source — an op is a vaelii.core
fn with the KB supplied — so a wrapper here is that fn's own spelling, bare or
!-marked exactly as vaelii.core spells it, at its own arities with kb replaced by
conn. It is generated at build time rather than macroexpanded from serve/ops,
because requiring the table would pull the engine, jetty and reitit onto the classpath
of a namespace whose whole point is not needing them. client_surface_test compares
this file against what the generator would write now, so an op added to the daemon
fails the suite until the wrapper is written.
A thin EDN-over-HTTP client for the vaelii daemon (`vaelii.host.serve`). Runs no
engine: it POSTs `{:op :args}` and reads the result back, over JDK `java.net.http`
(no dependency — JDK 21 ships it).
Every call threads an **explicit connection handle** as its first argument —
`(query conn '(dog ?x) 'Ctx)` — the network mirror of `vaelii.core`'s explicit-`kb`
API. A `conn` from `client` holds a reusable `HttpClient`; no socket opens until a
call. A daemon reply of `{:ok false}` becomes an `ex-info` carrying the daemon's
`:error`, `:type` and `:status`, so a remote naming/disjointness refusal surfaces like
a local one and the coarse client-fault/server-fault split the status carries is
readable without writing the request by hand.
**The bearer token rides on the request the daemon requires it on**: the `conn`
carries it (`VAELII_API_TOKEN` unless `:token` says otherwise) and every call sets one
more header on the builder it was already using. No dependency, no client state, and
the `conn` is still a map you can read.
**One wrapper per op, and they are generated** (`vaelii.regen-client`, `lein
regen-client`). The daemon's op table is the single source — an op is a `vaelii.core`
fn with the KB supplied — so a wrapper here is that fn's own spelling, bare or
`!`-marked exactly as `vaelii.core` spells it, at its own arities with `kb` replaced by
`conn`. It is generated at *build* time rather than macroexpanded from `serve/ops`,
because requiring the table would pull the engine, jetty and reitit onto the classpath
of a namespace whose whole point is not needing them. `client_surface_test` compares
this file against what the generator would write now, so an op added to the daemon
fails the suite until the wrapper is written.The CxCore ontology — Vaelii's vocabulary context. It defines and
documents the core predicates the engine interprets, as sentexes in CxCore:
the special-predicate surface (types/contexts, arg, disjoint, the set/*Rule
wrappers, the predicate metadata, negation, ist, the evaluables) and the
predicate meta-ontology. Documentation rides on comment sentexes,
(comment <term> "...") — ordinary sentexes (stored, indexed, queryable) — so the
KB documents itself in its own representation.
The content is a KB file, resources/kb/CxCore.txt (read by
vaelii.host.seed); this namespace loads it and reads the docs back.
CxCore is the spindle head: the root every context sees, and the only
layer a core-only KB has. The layers below it — the definitional upper
contexts (between Core and Universe) and the theory middle contexts (between
Universe and Well) — are the starter's, not the core KB's, and each wires itself
into the spindle in its own KB file (see vaelii.host.starter).
The CxCore ontology — Vaelii's vocabulary context. It defines and documents the core predicates the engine interprets, as sentexes in CxCore: the special-predicate surface (types/contexts, arg, disjoint, the `set/*Rule` wrappers, the predicate metadata, negation, `ist`, the evaluables) and the predicate meta-ontology. Documentation rides on `comment` sentexes, `(comment <term> "...")` — ordinary sentexes (stored, indexed, queryable) — so the KB documents itself in its own representation. The content is a KB file, `resources/kb/CxCore.txt` (read by vaelii.host.seed); this namespace loads it and reads the docs back. CxCore is the spindle **head**: the root every context sees, and the only layer a core-only KB has. The layers below it — the definitional `upper` contexts (between Core and Universe) and the theory `middle` contexts (between Universe and Well) — are the starter's, not the core KB's, and each wires itself into the spindle in its own KB file (see vaelii.host.starter).
A sentence in English, composed from what the KB already says about its own vocabulary rather than generated.
The read path is the one with no verifier. Nothing in the engine can say that an
English sentence describing (genl penguin bird) is wrong, so a fluent gloss is a way
to teach a reader something false through their only window onto the formal content —
which makes reading the more dangerous direction here, not the safer one. The defence
is to not write prose at all where the KB has already written it.
The vocabulary documents itself: every shipped predicate carries a comment sentex,
and those comments are written in a shape that is already a template —
(comment eats "(eats ?animal ?food) means that ?animal takes ?food as nourishment. …")
(comment genl "(genl ?subtype ?supertype) means that every ?subtype is a ?supertype. …")
a signature naming the argument positions with variables, then a clause saying what
the predicate means in those names. So glossing (eats Muffet kibble) is not a
generation problem: read eats's comment, take its first clause, substitute the actual
arguments for the signature's variables.
The variables are why it reads: a parameter spelled ?animal cannot be mistaken for an
individual the way Animal can, and because the name carries the sort, the clause
after it needs no sortal noun to lean on — so what substitutes is the sentence a reader
wants rather than one with place Paris in it. Everything past that first clause is
documentation for a reader, not template: how the predicate is used, what it is not,
and what the KB does with it.
A signature written with plain capitalized words and a colon — (eats Animal Food): Animal eats Food — is read the same way, since an imported vocabulary spells its own
comments and they are not ours to rewrite.
This is why the composer is a lookup and a substitution rather than a table of hand-written patterns: adding a predicate with a documented signature gives it a gloss for free, and a comment edited to say something else changes the gloss with it. Of the 328 shipped comments, 210 carry a signature; the 118 that do not are nouns — 100 types and 18 individuals (the units, the dimensions, the three signs) — which need none, because a type gloss is "X is a dog" and the comment is the apposition after it.
What the composition rate does not measure is whether a gloss is worth reading. It
helps a reader where the predicate name is opaque — genl glossed as "Every dog is an
animal" teaches a reader what genl means — and adds nothing where the predicate is
already an English verb.
A term with no comment degrades to naming the term. It does not invent a
description, because an invented description is exactly the failure this exists to
prevent, and a reader who sees the bare name has lost nothing they were entitled to.
Every result carries :source saying which it got:
:composed every literal came from a comment :partial some did; the rest are named :named nothing to compose from — the terms, in a frame
The formal sentence is never replaced by the gloss — that is the caller's contract, and
docs/web.md states it for the browser.
A sentence in English, **composed** from what the KB already says about its own
vocabulary rather than generated.
The read path is the one with no verifier. Nothing in the engine can say that an
English sentence describing `(genl penguin bird)` is wrong, so a fluent gloss is a way
to teach a reader something false through their only window onto the formal content —
which makes reading the more dangerous direction here, not the safer one. The defence
is to not write prose at all where the KB has already written it.
## The comment is the template
The vocabulary documents itself: every shipped predicate carries a `comment` sentex,
and those comments are written in a shape that is already a template —
(comment eats "(eats ?animal ?food) means that ?animal takes ?food as nourishment. …")
(comment genl "(genl ?subtype ?supertype) means that every ?subtype is a ?supertype. …")
a **signature** naming the argument positions with variables, then a clause saying what
the predicate means *in those names*. So glossing `(eats Muffet kibble)` is not a
generation problem: read `eats`'s comment, take its first clause, substitute the actual
arguments for the signature's variables.
The variables are why it reads: a parameter spelled `?animal` cannot be mistaken for an
individual the way `Animal` can, and because the *name* carries the sort, the clause
after it needs no sortal noun to lean on — so what substitutes is the sentence a reader
wants rather than one with `place Paris` in it. Everything past that first clause is
documentation for a reader, not template: how the predicate is used, what it is not,
and what the KB does with it.
A signature written with plain capitalized words and a colon — `(eats Animal Food):
Animal eats Food` — is read the same way, since an imported vocabulary spells its own
comments and they are not ours to rewrite.
This is why the composer is a lookup and a substitution rather than a table of
hand-written patterns: adding a predicate with a documented signature gives it a gloss
for free, and a comment edited to say something else changes the gloss with it. Of the
328 shipped comments, 210 carry a signature; the 118 that do not are nouns — 100 types
and 18 individuals (the units, the dimensions, the three signs) — which need none,
because a type gloss is "X is a dog" and the comment is the apposition after it.
What the composition rate does **not** measure is whether a gloss is worth reading. It
helps a reader where the predicate name is opaque — `genl` glossed as "Every dog is an
animal" teaches a reader what `genl` means — and adds nothing where the predicate is
already an English verb.
## What it will not do
A term with no comment **degrades to naming the term**. It does not invent a
description, because an invented description is exactly the failure this exists to
prevent, and a reader who sees the bare name has lost nothing they were entitled to.
Every result carries `:source` saying which it got:
:composed every literal came from a comment
:partial some did; the rest are named
:named nothing to compose from — the terms, in a frame
The formal sentence is never replaced by the gloss — that is the caller's contract, and
`docs/web.md` states it for the browser.The HTTP guards both servers hold to — vaelii.browser.web (the browser) and
vaelii.host.serve (the daemon).
The browser authenticates nobody and the daemon only when a token is set
(api-token), and both bind loopback for that reason. Loopback is what makes the
checks here necessary rather than sufficient: a browser running on the same machine
is a local client, so "only this machine may reach it" does not mean "only this
machine's owner may drive it". Two attacks follow from that, and each guard below
closes one.
Cross-site request forgery. Any page the operator visits can fetch a loopback
URL. same-origin? rejects the write whenever the browser stamps Origin.
edn-body? closes the case where it does not: application/edn is not a
CORS-simple content type, so a browser must preflight it, and a server answering
no CORS headers fails that preflight before the request is ever sent.
DNS rebinding. same-origin? compares Origin against the request's own
Host, so an attacker controlling both — a domain that re-resolves to 127.0.0.1
once the page is loaded — satisfies it. host-allowed? is the check that does not
fold, because the Host header must then name the interface the server was actually
started on.
The HTTP guards both servers hold to — `vaelii.browser.web` (the browser) and `vaelii.host.serve` (the daemon). The browser authenticates nobody and the daemon only when a token is set (`api-token`), and both bind loopback for that reason. Loopback is what makes the checks here necessary rather than sufficient: a browser running on the same machine *is* a local client, so "only this machine may reach it" does not mean "only this machine's owner may drive it". Two attacks follow from that, and each guard below closes one. **Cross-site request forgery.** Any page the operator visits can `fetch` a loopback URL. `same-origin?` rejects the write whenever the browser stamps `Origin`. `edn-body?` closes the case where it does not: `application/edn` is not a CORS-*simple* content type, so a browser must preflight it, and a server answering no CORS headers fails that preflight before the request is ever sent. **DNS rebinding.** `same-origin?` compares `Origin` against the request's own `Host`, so an attacker controlling both — a domain that re-resolves to 127.0.0.1 once the page is loaded — satisfies it. `host-allowed?` is the check that does not fold, because the `Host` header must then name the interface the server was actually started on.
Synthesize a knowledge base of a chosen shape.
The two other kinds of KB are given: a shipped ontology (vaelii.host.starter) is
fixed content, and an imported corpus (vaelii.impl.io.import, or a translated one)
is whatever the source says. Neither lets you ask what happens at ten times the
rules, and that is the question a scale or behaviour measurement is made of. So this
namespace generates a KB from a handful of numbers — how many types, individuals,
predicates, facts and rules, how the rules split forward/backward, how many of them are
defeasible — and each number is a knob the browser renders as a slider (knobs).
Two properties make a generated KB usable as a measurement rather than as noise:
plan's three draw streams owns a java.util.Random
seeded from the plan seed and its own constant (stream-seeds), so the same
parameters give the same KB whichever order a reader realizes the streams in — a
shape can be reproduced from the numbers alone, and a run compared against a rerun.plan is pure — the whole KB as data, nothing asserted. load-into asserts it,
reporting progress through an optional :on-progress callback (which may throw to
cancel the load, the flag vaelii.browser.catalog cancels on).
Synthesize a knowledge base of a chosen **shape**. The two other kinds of KB are given: a shipped ontology (`vaelii.host.starter`) is fixed content, and an imported corpus (`vaelii.impl.io.import`, or a translated one) is whatever the source says. Neither lets you ask *what happens at ten times the rules*, and that is the question a scale or behaviour measurement is made of. So this namespace generates a KB from a handful of numbers — how many types, individuals, predicates, facts and rules, how the rules split forward/backward, how many of them are defeasible — and each number is a knob the browser renders as a slider (`knobs`). Two properties make a generated KB usable as a measurement rather than as noise: * **Deterministic.** Each of `plan`'s three draw streams owns a `java.util.Random` seeded from the plan seed and its own constant (`stream-seeds`), so the same parameters give the same KB whichever order a reader realizes the streams in — a shape can be reproduced from the numbers alone, and a run compared against a rerun. * **Stratified.** Predicates are split into layers: facts populate layer 0, and a rule concluding a layer-k predicate draws its antecedents only from layers below k. The rule set is therefore acyclic, so forward chaining cascades base → derived → further-derived and terminates, instead of the runaway recursion a rule set wired at random produces. Individuals and predicates are Zipf-sampled, so the corpus has hot terms and a long tail like a real one rather than a uniform smear. `plan` is pure — the whole KB as data, nothing asserted. `load-into` asserts it, reporting progress through an optional `:on-progress` callback (which may throw to cancel the load, the flag `vaelii.browser.catalog` cancels on).
The browser editor's line format: a stored sentex as the sentence its author would type back in.
The editor seeds its textarea with these sentences and diffs the text it gets back
against them by content, so a sentence spelled two ways would turn an untouched line
into a retract plus an assert of the same fact. It lives here rather than in the
browser because spelling a rule's wrappers reads the rule record's slots
(rules/rewrap-sentex), and the browser requires no vaelii.impl namespace.
The browser editor's line format: a stored sentex as the sentence its author would type back in. The editor seeds its textarea with these sentences and diffs the text it gets back against them **by content**, so a sentence spelled two ways would turn an untouched line into a retract plus an assert of the same fact. It lives here rather than in the browser because spelling a rule's wrappers reads the rule record's slots (`rules/rewrap-sentex`), and the browser requires no `vaelii.impl` namespace.
Ontology KB files: declarative content held as plain text on the classpath rather than as code.
A KB file is a list of ordinary vaelii sentences — one s-expression each, with
;; line comments and blank lines allowed — named for the context its sentences
assert into, and grouped term-centrically: every sentence about a vocabulary
term sits together, and the terms run in natural sort order. A rule is just a
sentence carrying an implies / set/*Rule / exceptWhen wrapper.
The format itself — reader and writer both — is vaelii.impl.io.text, which is
where its one non-sentence spelling lives ((set/monotonic S), the known-true class)
and what vaelii.core/export-text! writes. What is here is the classpath side: the
shallow tree under resources/kb/ and how a layer's files are discovered in it.
The files live under resources/kb/, in a shallow tree that mirrors the context
spindle:
kb/CxCore.txt the vocabulary head (see vaelii.host.core-context)
kb/upper/<C>.txt definitional layers, between Core and Universe
kb/middle/<C>.txt theory layers, between Universe and Well
The file name is the context; the sub-directory is the layer. Only the layer a
caller names is discovered, so a sibling directory under kb/ that names no layer here
is not loaded: kb/koinii/ is one, an application's own context files, which that
application loads for itself. What stays in
code (vaelii.host.starter) is the order the files load in and the handful of
genuinely computed assertions. Sentences read with clojure.edn, so a KB file is
data and can never run code.
Ontology KB files: declarative content held as **plain text on the classpath**
rather than as code.
A KB file is a list of ordinary vaelii sentences — one s-expression each, with
`;;` line comments and blank lines allowed — named for the context its sentences
assert into, and grouped **term-centrically**: every sentence about a vocabulary
term sits together, and the terms run in natural sort order. A rule is just a
sentence carrying an `implies` / `set/*Rule` / `exceptWhen` wrapper.
**The format itself — reader and writer both — is `vaelii.impl.io.text`**, which is
where its one non-sentence spelling lives (`(set/monotonic S)`, the known-true class)
and what `vaelii.core/export-text!` writes. What is here is the *classpath* side: the
shallow tree under `resources/kb/` and how a layer's files are discovered in it.
The files live under `resources/kb/`, in a shallow tree that mirrors the context
spindle:
kb/CxCore.txt the vocabulary head (see vaelii.host.core-context)
kb/upper/<C>.txt definitional layers, between Core and Universe
kb/middle/<C>.txt theory layers, between Universe and Well
The file *name* is the context; the sub-directory is the layer. Only the layer a
caller names is discovered, so a sibling directory under `kb/` that names no layer here
is not loaded: `kb/koinii/` is one, an application's own context files, which that
application loads for itself. What stays in
**code** (vaelii.host.starter) is the *order* the files load in and the handful of
genuinely computed assertions. Sentences read with `clojure.edn`, so a KB file is
data and can never run code.Headless EDN-over-HTTP daemon: one JVM owns one KB and serves it to remote clients
(vaelii.host.client). A thin reitit-ring + jetty layer over vaelii.core, the
network dual of the in-process API.
Wire format is EDN. A sentence is a symbol s-expression — (dog Muffet), ?x,
(genl dog animal) — which EDN round-trips losslessly; JSON would mangle the symbols.
The body of every call is {:op <keyword> :args [...]}, and the reply is
{:ok true :result …} or {:ok false :error "…"}. EDN is read with
clojure.edn/read-string (never clojure.core/read-string), so an untrusted body
cannot evaluate code — EDN has no reader-eval.
A refusal's :type is a plain keyword — :body-too-large, :not-edn,
:cross-origin, :bad-host — and so is the :type an engine ex-info carries
through. The protocol is what a client written against another build discriminates
on, so it cannot be qualified by the namespace that happens to serve it: a
::-qualified keyword names this namespace, and a client matching on it would be
matching on where the daemon's code lives.
The daemon is the single writer (docs/storage.md, the single-writer contract): it owns the one process allowed to mutate the store, so it serializes every op through one monitor. Concurrent client writes therefore apply one at a time and cannot interleave; reads pay the same lock, which is conservative but keeps the contract simple.
Only the allowlisted ops are reachable (ops). Each is a vaelii.core fn with
the KB supplied by the daemon — the client sends only the op and the remaining args —
so no client can reach an arbitrary var. Sentex records in a result are projected to
plain maps before they hit the wire (the sentex-map contract), so the client reads
them back without the impl record class.
The change feed is the one thing that is not a vaelii.core fn (feed-ops), and
it is a table of its own for that reason: core/watch takes a callback, so what a
remote caller holds open instead is a subscription with a cursor — :watch,
:poll, :unwatch, :watchers, over the per-handler registry app builds
(vaelii.host.subscribe, docs/feed.md). A :poll that waits runs outside the
monitor; everything else about them is an ordinary EDN op.
One shared bearer token authenticates the caller. With VAELII_API_TOKEN set
(guard/api-token), every request presents Authorization: Bearer <token> or is
answered 401 with a WWW-Authenticate: Bearer challenge; GET /health is the one
route that answers without it. One token for the process, not a session and not an
identity — per-caller identity is a reverse proxy's job, and this is the check that
has to exist below it. Binding anything but loopback requires a token (-main
refuses to start otherwise); on the loopback default it is optional, and a daemon
without one is drivable by every process on the machine.
vaelii.host.guard covers what a token does not, and matters most on the open
loopback daemon: POST /op requires Content-Type: application/edn, refuses a
cross-origin Origin, and answers only to a Host naming the interface it was
started on. Together those stop a page the operator happens to visit from driving
the KB over loopback — which binding to loopback alone does not.
Headless EDN-over-HTTP daemon: one JVM owns one KB and serves it to remote clients
(`vaelii.host.client`). A thin reitit-ring + jetty layer over `vaelii.core`, the
network dual of the in-process API.
**Wire format is EDN.** A sentence is a symbol s-expression — `(dog Muffet)`, `?x`,
`(genl dog animal)` — which EDN round-trips losslessly; JSON would mangle the symbols.
The body of every call is `{:op <keyword> :args [...]}`, and the reply is
`{:ok true :result …}` or `{:ok false :error "…"}`. EDN is read with
`clojure.edn/read-string` (never `clojure.core/read-string`), so an untrusted body
cannot evaluate code — EDN has no reader-eval.
**A refusal's `:type` is a plain keyword** — `:body-too-large`, `:not-edn`,
`:cross-origin`, `:bad-host` — and so is the `:type` an engine `ex-info` carries
through. The protocol is what a client written against another build discriminates
on, so it cannot be qualified by the namespace that happens to serve it: a
`::`-qualified keyword names *this* namespace, and a client matching on it would be
matching on where the daemon's code lives.
**The daemon is the single writer** (docs/storage.md, the single-writer contract): it
owns the one process allowed to mutate the store, so it serializes every op through
one monitor. Concurrent client writes therefore apply one at a time and cannot
interleave; reads pay the same lock, which is conservative but keeps the contract
simple.
**Only the allowlisted ops are reachable** (`ops`). Each is a `vaelii.core` fn with
the KB supplied by the daemon — the client sends only the op and the remaining args —
so no client can reach an arbitrary var. Sentex records in a result are projected to
plain maps before they hit the wire (the `sentex`-map contract), so the client reads
them back without the `impl` record class.
**The change feed is the one thing that is not a `vaelii.core` fn** (`feed-ops`), and
it is a table of its own for that reason: `core/watch` takes a callback, so what a
remote caller holds open instead is a subscription with a **cursor** — `:watch`,
`:poll`, `:unwatch`, `:watchers`, over the per-handler registry `app` builds
(`vaelii.host.subscribe`, docs/feed.md). A `:poll` that waits runs **outside** the
monitor; everything else about them is an ordinary EDN op.
**One shared bearer token authenticates the caller.** With `VAELII_API_TOKEN` set
(`guard/api-token`), every request presents `Authorization: Bearer <token>` or is
answered 401 with a `WWW-Authenticate: Bearer` challenge; `GET /health` is the one
route that answers without it. One token for the process, not a session and not an
identity — per-caller identity is a reverse proxy's job, and this is the check that
has to exist below it. Binding anything but loopback **requires** a token (`-main`
refuses to start otherwise); on the loopback default it is optional, and a daemon
without one is drivable by every process on the machine.
`vaelii.host.guard` covers what a token does not, and matters most on the open
loopback daemon: `POST /op` requires `Content-Type: application/edn`, refuses a
cross-origin `Origin`, and answers only to a `Host` naming the interface it was
started on. Together those stop a page the operator happens to visit from driving
the KB over loopback — which binding to loopback alone does not.Bring a KB's shipped spindle up to the running engine's.
A KB stores the starter ontology it was built with, and the engine that opens it later
ships its own: a strength marked set/monotonic, a vocabulary term added to CxCore, a
disjointness the ontology stopped stating. sync-spindle! makes the KB state what this
engine ships, context by context, and leaves everything else alone.
What is shipped is what starter/load-into produces, read off a scratch in-memory KB
it is loaded into — not the text of the files. The two differ: genlCx edges are stored
in CxUniverse whichever file states them, and the closing unary_predicate batch is
stated by no file. Each premise is attributed to the file whose load stored it.
Which contexts are synced exactly: CxCore and the kb/upper/ and kb/middle/
contexts. These are the engine's, so a premise there that the engine does not ship is
retracted, a strength that differs is restated, and a missing one is asserted. A
context an author adds to the spindle — wired between CxCore and CxUniverse, say — is
not one of them and is not read.
Which are only added to: the collectors (kb/ root files, CxUniverse today) and any
other context a shipped file's content is stored in. A collector gathers what the
engine routes there from every context, so what it holds beyond the shipped content is
not the engine's to retract.
Only the layers a KB has. A file whose context holds nothing in the KB is not
loaded into it, and the collector files and the unary_predicate batch follow the upper
layer: a core-only KB stays core-only.
Comparison is by the form a KB file writes (text/premise-entries): a fact or rule
under its strength wrapper, an exceptWhen as the wrapper it was asserted as. A
strength the engine raised is only an assertion, since assert raises a held
premise's strength in place; one it lowered is the form retracted and the weaker one
asserted, since nothing lowers a strength in place.
One batch. The sync is one v/edit!: the additions first, then the retractions,
one settle. So the belief a retraction would sweep and the addition rebuild keeps its
witness through the batch, where retracting first tore the TMS down only to build it
back up. What cannot go in that order — a retraction of a record an addition lands on —
goes before it (sync-spindle!).
Bring a KB's shipped spindle up to the running engine's. A KB stores the starter ontology it was built with, and the engine that opens it later ships its own: a strength marked `set/monotonic`, a vocabulary term added to CxCore, a disjointness the ontology stopped stating. `sync-spindle!` makes the KB state what this engine ships, context by context, and leaves everything else alone. **What is shipped** is what `starter/load-into` produces, read off a scratch in-memory KB it is loaded into — not the text of the files. The two differ: `genlCx` edges are stored in CxUniverse whichever file states them, and the closing `unary_predicate` batch is stated by no file. Each premise is attributed to the file whose load stored it. **Which contexts are synced exactly**: CxCore and the `kb/upper/` and `kb/middle/` contexts. These are the engine's, so a premise there that the engine does not ship is retracted, a strength that differs is restated, and a missing one is asserted. A context an author adds to the spindle — wired between CxCore and CxUniverse, say — is not one of them and is not read. **Which are only added to**: the collectors (`kb/` root files, CxUniverse today) and any other context a shipped file's content is stored in. A collector gathers what the engine routes there from every context, so what it holds beyond the shipped content is not the engine's to retract. **Only the layers a KB has.** A file whose context holds nothing in the KB is not loaded into it, and the collector files and the `unary_predicate` batch follow the upper layer: a core-only KB stays core-only. Comparison is by the form a KB file writes (`text/premise-entries`): a fact or rule under its strength wrapper, an `exceptWhen` as the wrapper it was asserted as. A strength the engine **raised** is only an assertion, since `assert` raises a held premise's strength in place; one it **lowered** is the form retracted and the weaker one asserted, since nothing lowers a strength in place. **One batch.** The sync is one `v/edit!`: the additions first, then the retractions, one settle. So the belief a retraction would sweep and the addition rebuild keeps its witness through the batch, where retracting first tore the TMS down only to build it back up. What cannot go in that order — a retraction of a record an addition lands on — goes before it (`sync-spindle!`).
A starter common-sense KB: a documented, schema-only upper + middle ontology. It loads the CxCore vocabulary (vaelii.host.core-context), then the starter's own contexts, each a KB file on the classpath under resources/kb/:
genl. Split by domain, one context each:
A spindle is three layers — a head every member sees, members that see the head and
not each other, and a collector that sees every member — and the topology is two of
them stacked, most general (top) to most specific (bottom): CxCore heads the
upper spindle, whose members are kb/upper/ and whose collector is CxUniverse, and
CxUniverse heads the middle spindle, whose members are kb/middle/ and whose
collector is CxWell. Each member file wires itself to its own head and collector, so
the topology is data. No cast and no contingent facts ship: the starter is a schema, and
contingent data (a cast, worked examples, the Aesop fables) belongs below CxWell
and lives in the tests that need it.
The unit table is the one place individuals ship, and it applies that rule rather than excepting itself from it: a minute is sixty seconds by stipulation, so the factor is vocabulary and not a measurement anybody took. CxMeasure.txt states the test it holds a unit to.
What stays in code here is the order the layers load in — loading order is logic,
the definitional layer must precede the theories that reason over it — and the one
computed batch (every type is also a unary_predicate). Within a layer, every
context file present is loaded (discovered from the classpath), so adding a KB is
dropping a file in kb/upper/ or kb/middle/, no code change. seed/root-contexts
discovers the top-level collector files (kb/Cx<Name>.txt other than CxCore.txt,
today CxUniverse.txt) from the classpath as well.
A starter common-sense KB: a documented, **schema-only** upper + middle ontology.
It loads the CxCore vocabulary (vaelii.host.core-context), then the starter's own
contexts, each a KB file on the classpath under resources/kb/:
* upper (definitional — between Core and Universe): what things *are*, always
true, like `genl`. Split by domain, one context each:
- CxAbstract.txt — the abstract type skeleton (physical/intangible and
their kinds) + the structural relations partOf/locatedIn.
- CxOrganism.txt — the biological taxonomy + its disjointness.
- CxLife.txt — the organism relations (parentOf, siblingOf, flies,
mortal, birthYearOf, olderThan, …) with arg + metadata.
- CxSociety.txt — the social relations (marriedTo, likes, owns).
- CxMeasure.txt — the theory of measurement: the two measure terms, the
dimensionOf/conversionFactor table with the units that
fill it, the comparisons, weightOf / heightOf, and the
sign vocabulary for the quantities nobody has a figure
for (signOf / trendOf / the qualitative* arithmetic).
- CxSpace.txt — qualitative space, four independent calculi: RCC-8
topology (eight base + six derived), cardinal direction
(nine + four), relative direction over a frame's own axes
(nine + four, the frame being the context), and
qualitative distance (seven + three).
- CxTime.txt — qualitative time: Allen's interval relations (thirteen
base + seven derived), the point algebra over instants,
the three calendar constructors and the InstantFn moment
a calendar term's startOf and endOf are computed as, plus
the length / totalDuration / overlapDuration vocabulary
the arithmetic computes over.
* middle (theory — between Universe and Well): how the definitional things
*interrelate*, where several overlapping theories can coexist.
- CxAnatomy.txt — what kinds of thing have what kinds of part.
- CxBiology.txt — birds fly by default except penguins; organisms
are mortal; flight enables travel; sleep is what the
theory is willing to assume.
- CxChange.txt — a simple event calculus: a state persists until an
event ends it, so holdsAt is inertia over what
initiates and terminates say.
- CxKinship.txt — grandparentOf, ancestorOf, olderThan, and parenthood
from maternity and paternity.
- CxMereology.txt — a part is located where its whole is; owning a whole
entails owning its parts.
- CxSize.txt — comparative size: stated between kinds, computed
between objects from their measures.
- CxSocial.txt — what acquaintance follows from; employment as one way
of belonging.
A spindle is three layers — a head every member sees, members that see the head and
not each other, and a collector that sees every member — and the topology is two of
them stacked, most general (top) to most specific (bottom): CxCore heads the
upper spindle, whose members are `kb/upper/` and whose collector is CxUniverse, and
CxUniverse heads the middle spindle, whose members are `kb/middle/` and whose
collector is CxWell. Each member file wires itself to its own head and collector, so
the topology is data. **No cast and no contingent facts ship**: the starter is a schema, and
contingent data (a cast, worked examples, the Aesop fables) belongs below CxWell
and lives in the tests that need it.
The unit table is the one place individuals ship, and it applies that rule rather
than excepting itself from it: a minute is sixty seconds by stipulation, so the
factor is vocabulary and not a measurement anybody took. CxMeasure.txt states
the test it holds a unit to.
What stays in code here is the *order the layers* load in — loading order is logic,
the definitional layer must precede the theories that reason over it — and the one
computed batch (every type is also a unary_predicate). Within a layer, every
context file present is loaded (discovered from the classpath), so adding a KB is
dropping a file in kb/upper/ or kb/middle/, no code change. `seed/root-contexts`
discovers the top-level collector files (`kb/Cx<Name>.txt` other than `CxCore.txt`,
today `CxUniverse.txt`) from the classpath as well.The change feed with a cursor where the in-process one has a callback — the daemon-side state a remote caller holds a feed open against.
core/watch takes a function, and a function does not cross an EDN wire (the same
wall :export's :on-progress hits). So the wire's half of the feed is not the
callback marshalled somehow; it is the one thing a request/response protocol can
carry, which is state with a cursor: the daemon registers an ordinary listener of
its own, that listener files each event into a bounded ring, and a caller reads the
ring forward from where it left off. Three ops — register, read, drop — every one of
them EDN in and EDN out, so the guards, the client and the error taxonomy that already
exist carry it unchanged (docs/operations.md).
A cursor counts events, not handles. It starts at 0 when the subscription is
registered and advances by one per delivered event, so a caller compares nothing and
stores one integer. poll answers the events past the cursor it was handed and the
cursor to send next time.
The ring is bounded, and falling off it is said out loud. A subscriber that stops
reading must not grow the daemon's heap, so the ring keeps max-events and drops the
oldest past it — and the count of what it dropped is reported as :lagged on the
next poll. That number is the whole reason this is usable: a feed with a silent gap
is strictly worse than polling, because the caller believes it is current and is not.
:lagged is present on every reply, zero and all, so a client that forgets to read it
is a client that cannot have one.
A token that names no subscription is refused, never answered empty. The same
argument: a reaped, dropped or invented token answering {:events []} is a feed that
has silently stopped. :unknown-subscription says so.
What a subscription costs the daemon, and what bounds it. One listener on the
KB's feed and one ring of at most max-events events; max-subscriptions of those at
once, and one that nobody has polled inside idle-ms is reaped at the next call.
Nothing here authenticates the caller — that is the bearer token's job, one layer out
(vaelii.host.serve) — but heap a stranger can allocate wants a ceiling whether or
not it is authenticated, and the reap is what keeps an abandoned subscription from
holding a slot against a live one.
The wait happens here, outside the daemon's monitor. A long poll parks on the
subscription's own signal object, so a writer serialized behind serve's one monitor
runs to completion while a poll is parked — the feature is about liveness, and a
parked poll that blocked every writer would be a global stall wearing its name. The
writing thread's only cost is the swap that files the event and a notifyAll on a
monitor no poller holds for longer than a compare.
The three entry points are spelled without ! for core/watch's reason: nothing here
destroys stored knowledge (docs/api.md). See docs/feed.md, "Across the wire".
The change feed with a **cursor** where the in-process one has a callback — the
daemon-side state a remote caller holds a feed open against.
`core/watch` takes a function, and a function does not cross an EDN wire (the same
wall `:export`'s `:on-progress` hits). So the wire's half of the feed is not the
callback marshalled somehow; it is the one thing a request/response protocol can
carry, which is **state with a cursor**: the daemon registers an ordinary listener of
its own, that listener files each event into a bounded ring, and a caller reads the
ring forward from where it left off. Three ops — register, read, drop — every one of
them EDN in and EDN out, so the guards, the client and the error taxonomy that already
exist carry it unchanged (docs/operations.md).
**A cursor counts events, not handles.** It starts at 0 when the subscription is
registered and advances by one per delivered event, so a caller compares nothing and
stores one integer. `poll` answers the events past the cursor it was handed and the
cursor to send next time.
**The ring is bounded, and falling off it is said out loud.** A subscriber that stops
reading must not grow the daemon's heap, so the ring keeps `max-events` and drops the
oldest past it — and the *count* of what it dropped is reported as `:lagged` on the
next poll. That number is the whole reason this is usable: a feed with a silent gap
is strictly worse than polling, because the caller believes it is current and is not.
`:lagged` is present on every reply, zero and all, so a client that forgets to read it
is a client that cannot have one.
**A token that names no subscription is refused, never answered empty.** The same
argument: a reaped, dropped or invented token answering `{:events []}` is a feed that
has silently stopped. `:unknown-subscription` says so.
**What a subscription costs the daemon, and what bounds it.** One listener on the
KB's feed and one ring of at most `max-events` events; `max-subscriptions` of those at
once, and one that nobody has polled inside `idle-ms` is reaped at the next call.
Nothing here authenticates the caller — that is the bearer token's job, one layer out
(`vaelii.host.serve`) — but heap a stranger can allocate wants a ceiling whether or
not it is authenticated, and the reap is what keeps an abandoned subscription from
holding a slot against a live one.
**The wait happens here, outside the daemon's monitor.** A long poll parks on the
subscription's own signal object, so a writer serialized behind `serve`'s one monitor
runs to completion while a poll is parked — the feature is about liveness, and a
parked poll that blocked every writer would be a global stall wearing its name. The
writing thread's only cost is the swap that files the event and a `notifyAll` on a
monitor no poller holds for longer than a compare.
The three entry points are spelled without `!` for `core/watch`'s reason: nothing here
destroys stored knowledge (docs/api.md). See docs/feed.md, "Across the wire".cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |