A command-line driver for a KB — the shell dual of the in-process API, launched with
lein run -m vaelii.host.cli <cmd> <args…>. It runs the engine in-process (no
daemon); to talk to a running daemon use vaelii.host.client instead.
lein run -m vaelii.host.cli assert '(dog Muffet)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli query '(dog ?x)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli why 3 --dir /tmp/kb lein run -m vaelii.host.cli export /tmp/dump --dir /tmp/kb lein run -m vaelii.host.cli repl --starter # interactive, starter schema lein cli help # every command and what it takes
help is a word rather than only a flag because Leiningen answers lein cli --help
itself, printing the alias expansion — the flag never reaches this namespace through
the alias, though it does through the full lein run -m vaelii.host.cli --help.
Backend. --dir <path> opens the store there under the backend its files were
written by (v/store-backend), or a new durable :disk-log store when it holds none —
recovered on open, so a fact asserted in one invocation is there in the next. A --dir
whose parent does not exist is refused (open-kb-from) rather than created.
upgrade opens a store, brings its reasoning image and index image up to this build, and
closes it (upgrade!); --verify recovers anyway and compares the two images.
With no --dir the KB is
in-memory and lives only for the process — useful for repl or a single compound
session, pointless across one-shot commands. --starter loads the shipped schema
(types, contexts, relation rules) so you can explore the ontology. --strength monotonic marks an assert or assert-rule known-true. export takes --variant records|records+index and --compression gzip|xz|none.
A flag belongs to the commands that read it (command-flags), and one carried by
a command that does not is refused rather than dropped — those three are the driver's
and go anywhere, the rest do not. A value it cannot honour is refused on the same
argument: --format texr and --depth twice name nothing.
stdout is the answer. err! keeps a refusal off it, and on-stderr keeps the
engine's own log lines off it too — Trove's console backend prints to *out*, which
here is what a script redirects. A refusal is one stderr line, error: [<:type>] <message> (refusal-line), so a script branches on the keyword and not on the prose.
One writer. A --dir KB takes the single-writer file lock (docs/storage.md), so
the CLI and a daemon cannot own the same directory at once — by design. diff and
upgrade open no KB of the run's (without-a-kb), so diff answers beside a daemon
that holds --dir.
A command-line driver for a KB — the shell dual of the in-process API, launched with `lein run -m vaelii.host.cli <cmd> <args…>`. It runs the engine in-process (no daemon); to talk to a running daemon use `vaelii.host.client` instead. lein run -m vaelii.host.cli assert '(dog Muffet)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli query '(dog ?x)' CxNaturalWorld --dir /tmp/kb lein run -m vaelii.host.cli why 3 --dir /tmp/kb lein run -m vaelii.host.cli export /tmp/dump --dir /tmp/kb lein run -m vaelii.host.cli repl --starter # interactive, starter schema lein cli help # every command and what it takes `help` is a word rather than only a flag because Leiningen answers `lein cli --help` itself, printing the alias expansion — the flag never reaches this namespace through the alias, though it does through the full `lein run -m vaelii.host.cli --help`. **Backend.** `--dir <path>` opens the store there under the backend its files were written by (`v/store-backend`), or a new durable `:disk-log` store when it holds none — recovered on open, so a fact asserted in one invocation is there in the next. A `--dir` whose parent does not exist is refused (`open-kb-from`) rather than created. `upgrade` opens a store, brings its reasoning image and index image up to this build, and closes it (`upgrade!`); `--verify` recovers anyway and compares the two images. With no `--dir` the KB is in-memory and lives only for the process — useful for `repl` or a single compound session, pointless across one-shot commands. `--starter` loads the shipped schema (types, contexts, relation rules) so you can explore the ontology. `--strength monotonic` marks an `assert` or `assert-rule` known-true. `export` takes `--variant records|records+index` and `--compression gzip|xz|none`. **A flag belongs to the commands that read it** (`command-flags`), and one carried by a command that does not is refused rather than dropped — those three are the driver's and go anywhere, the rest do not. A *value* it cannot honour is refused on the same argument: `--format texr` and `--depth twice` name nothing. **stdout is the answer.** `err!` keeps a refusal off it, and `on-stderr` keeps the engine's own log lines off it too — Trove's console backend prints to `*out*`, which here is what a script redirects. A refusal is one stderr line, `error: [<:type>] <message>` (`refusal-line`), so a script branches on the keyword and not on the prose. **One writer.** A `--dir` KB takes the single-writer file lock (docs/storage.md), so the CLI and a daemon cannot own the same directory at once — by design. `diff` and `upgrade` open no KB of the run's (`without-a-kb`), so `diff` answers beside a daemon that holds `--dir`.
A thin EDN-over-HTTP client for the vaelii daemon (vaelii.host.serve). Runs no
engine: it POSTs {:op :args} and reads the result back, over JDK java.net.http
(no dependency — JDK 21 ships it).
Every call threads an explicit connection handle as its first argument —
(query conn '(dog ?x) 'Ctx) — the network mirror of vaelii.core's explicit-kb
API. A conn from client holds a reusable HttpClient; no socket opens until a
call. A daemon reply of {:ok false} becomes an ex-info carrying the daemon's
:error, :type and :status, so a remote naming/disjointness refusal surfaces like
a local one and the coarse client-fault/server-fault split the status carries is
readable without writing the request by hand.
The bearer token rides on the request the daemon requires it on: the conn
carries it (VAELII_API_TOKEN unless :token says otherwise) and every call sets one
more header on the builder it was already using. No dependency, no client state, and
the conn is still a map you can read.
One wrapper per op, and they are generated (vaelii.regen-client, lein regen-client). The daemon's op table is the single source — an op is a vaelii.core
fn with the KB supplied — so a wrapper here is that fn's own spelling, bare or
!-marked exactly as vaelii.core spells it, at its own arities with kb replaced by
conn. It is generated at build time rather than macroexpanded from serve/ops,
because requiring the table would pull the engine, jetty and reitit onto the classpath
of a namespace whose whole point is not needing them. client_surface_test compares
this file against what the generator would write now, so an op added to the daemon
fails the suite until the wrapper is written.
A thin EDN-over-HTTP client for the vaelii daemon (`vaelii.host.serve`). Runs no
engine: it POSTs `{:op :args}` and reads the result back, over JDK `java.net.http`
(no dependency — JDK 21 ships it).
Every call threads an **explicit connection handle** as its first argument —
`(query conn '(dog ?x) 'Ctx)` — the network mirror of `vaelii.core`'s explicit-`kb`
API. A `conn` from `client` holds a reusable `HttpClient`; no socket opens until a
call. A daemon reply of `{:ok false}` becomes an `ex-info` carrying the daemon's
`:error`, `:type` and `:status`, so a remote naming/disjointness refusal surfaces like
a local one and the coarse client-fault/server-fault split the status carries is
readable without writing the request by hand.
**The bearer token rides on the request the daemon requires it on**: the `conn`
carries it (`VAELII_API_TOKEN` unless `:token` says otherwise) and every call sets one
more header on the builder it was already using. No dependency, no client state, and
the `conn` is still a map you can read.
**One wrapper per op, and they are generated** (`vaelii.regen-client`, `lein
regen-client`). The daemon's op table is the single source — an op is a `vaelii.core`
fn with the KB supplied — so a wrapper here is that fn's own spelling, bare or
`!`-marked exactly as `vaelii.core` spells it, at its own arities with `kb` replaced by
`conn`. It is generated at *build* time rather than macroexpanded from `serve/ops`,
because requiring the table would pull the engine, jetty and reitit onto the classpath
of a namespace whose whole point is not needing them. `client_surface_test` compares
this file against what the generator would write now, so an op added to the daemon
fails the suite until the wrapper is written.The upper ontology: every KB file under resources/kb/upper/, loaded into CxUniverse.
core.txt is the vocabulary — the special-predicate surface the engine interprets
(types/contexts, arg, disjoint, the set/*Rule wrappers, the predicate metadata,
negation, ist, the evaluables), the predicate meta-ontology and the top of the
ontology — and each other file is one domain (abstract, life, measure, organism,
reflection, society, space, time). Documentation rides on comment sentexes,
(comment <term> "...") — ordinary sentexes (stored, indexed, queryable) — so the
KB documents itself in its own representation.
CxUniverse is the root: every context the middle layer wires in sees it, and a KB built for vocabulary alone holds exactly this load. The middle contexts below it are the starter's (see vaelii.host.starter).
The upper ontology: every KB file under `resources/kb/upper/`, loaded into CxUniverse. `core.txt` is the vocabulary — the special-predicate surface the engine interprets (types/contexts, arg, disjoint, the `set/*Rule` wrappers, the predicate metadata, negation, `ist`, the evaluables), the predicate meta-ontology and the top of the ontology — and each other file is one domain (abstract, life, measure, organism, reflection, society, space, time). Documentation rides on `comment` sentexes, `(comment <term> "...")` — ordinary sentexes (stored, indexed, queryable) — so the KB documents itself in its own representation. CxUniverse is the root: every context the middle layer wires in sees it, and a KB built for vocabulary alone holds exactly this load. The middle contexts below it are the starter's (see vaelii.host.starter).
A sentence in English, composed from what the KB already says about its own vocabulary rather than generated.
The read path is the one with no verifier. Nothing in the engine can say that an
English sentence describing (genl penguin bird) is wrong, so a fluent gloss is a way
to teach a reader something false through their only window onto the formal content —
which makes reading the more dangerous direction here, not the safer one. The defence
is to not write prose at all where the KB has already written it.
The vocabulary documents itself: every shipped predicate carries a comment sentex,
and those comments are written in a shape that is already a template —
(comment eats "(eats ?animal ?food) means that ?animal takes ?food as nourishment. …")
(comment genl "(genl ?subtype ?supertype) means that every ?subtype is a ?supertype. …")
a signature naming the argument positions with variables, then a clause saying what
the predicate means in those names. So glossing (eats Muffet kibble) is not a
generation problem: read eats's comment, take its first clause, substitute the actual
arguments for the signature's variables.
The variables are why it reads: a parameter spelled ?animal cannot be mistaken for an
individual the way Animal can, and because the name carries the sort, the clause
after it needs no sortal noun to lean on — so what substitutes is the sentence a reader
wants rather than one with place Paris in it. Everything past that first clause is
documentation for a reader, not template: how the predicate is used, what it is not,
and what the KB does with it.
A signature written with plain capitalized words and a colon — (eats Animal Food): Animal eats Food — is read the same way, since an imported vocabulary spells its own
comments and they are not ours to rewrite.
This is why the composer is a lookup and a substitution rather than a table of hand-written patterns: adding a predicate with a documented signature gives it a gloss for free, and a comment edited to say something else changes the gloss with it. Of the 328 shipped comments, 210 carry a signature; the 118 that do not are nouns — 100 types and 18 individuals (the units, the dimensions, the three signs) — which need none, because a type gloss is "X is a dog" and the comment is the apposition after it.
What the composition rate does not measure is whether a gloss is worth reading. It
helps a reader where the predicate name is opaque — genl glossed as "Every dog is an
animal" teaches a reader what genl means — and adds nothing where the predicate is
already an English verb.
A term with no comment degrades to naming the term. It does not invent a
description, because an invented description is exactly the failure this exists to
prevent, and a reader who sees the bare name has lost nothing they were entitled to.
Every result carries :source saying which it got:
:composed every literal came from a comment :partial some did; the rest are named :named nothing to compose from — the terms, in a frame
The formal sentence is never replaced by the gloss — that is the caller's contract, and
docs/web.md states it for the browser.
A sentence in English, **composed** from what the KB already says about its own
vocabulary rather than generated.
The read path is the one with no verifier. Nothing in the engine can say that an
English sentence describing `(genl penguin bird)` is wrong, so a fluent gloss is a way
to teach a reader something false through their only window onto the formal content —
which makes reading the more dangerous direction here, not the safer one. The defence
is to not write prose at all where the KB has already written it.
## The comment is the template
The vocabulary documents itself: every shipped predicate carries a `comment` sentex,
and those comments are written in a shape that is already a template —
(comment eats "(eats ?animal ?food) means that ?animal takes ?food as nourishment. …")
(comment genl "(genl ?subtype ?supertype) means that every ?subtype is a ?supertype. …")
a **signature** naming the argument positions with variables, then a clause saying what
the predicate means *in those names*. So glossing `(eats Muffet kibble)` is not a
generation problem: read `eats`'s comment, take its first clause, substitute the actual
arguments for the signature's variables.
The variables are why it reads: a parameter spelled `?animal` cannot be mistaken for an
individual the way `Animal` can, and because the *name* carries the sort, the clause
after it needs no sortal noun to lean on — so what substitutes is the sentence a reader
wants rather than one with `place Paris` in it. Everything past that first clause is
documentation for a reader, not template: how the predicate is used, what it is not,
and what the KB does with it.
A signature written with plain capitalized words and a colon — `(eats Animal Food):
Animal eats Food` — is read the same way, since an imported vocabulary spells its own
comments and they are not ours to rewrite.
This is why the composer is a lookup and a substitution rather than a table of
hand-written patterns: adding a predicate with a documented signature gives it a gloss
for free, and a comment edited to say something else changes the gloss with it. Of the
328 shipped comments, 210 carry a signature; the 118 that do not are nouns — 100 types
and 18 individuals (the units, the dimensions, the three signs) — which need none,
because a type gloss is "X is a dog" and the comment is the apposition after it.
What the composition rate does **not** measure is whether a gloss is worth reading. It
helps a reader where the predicate name is opaque — `genl` glossed as "Every dog is an
animal" teaches a reader what `genl` means — and adds nothing where the predicate is
already an English verb.
## What it will not do
A term with no comment **degrades to naming the term**. It does not invent a
description, because an invented description is exactly the failure this exists to
prevent, and a reader who sees the bare name has lost nothing they were entitled to.
Every result carries `:source` saying which it got:
:composed every literal came from a comment
:partial some did; the rest are named
:named nothing to compose from — the terms, in a frame
The formal sentence is never replaced by the gloss — that is the caller's contract, and
`docs/web.md` states it for the browser.The HTTP guards both servers hold to — vaelii.browser.web (the browser) and
vaelii.host.serve (the daemon).
The browser authenticates nobody and the daemon only when a token is set
(api-token), and both bind loopback for that reason. Loopback is what makes the
checks here necessary rather than sufficient: a browser running on the same machine
is a local client, so "only this machine may reach it" does not mean "only this
machine's owner may drive it". Two attacks follow from that, and each guard below
closes one.
Cross-site request forgery. Any page the operator visits can fetch a loopback
URL. same-origin? rejects the write whenever the browser stamps Origin.
edn-body? closes the case where it does not: application/edn is not a
CORS-simple content type, so a browser must preflight it, and a server answering
no CORS headers fails that preflight before the request is ever sent.
DNS rebinding. same-origin? compares Origin against the request's own
Host, so an attacker controlling both — a domain that re-resolves to 127.0.0.1
once the page is loaded — satisfies it. host-allowed? is the check that does not
fold, because the Host header must then name the interface the server was actually
started on.
The HTTP guards both servers hold to — `vaelii.browser.web` (the browser) and `vaelii.host.serve` (the daemon). The browser authenticates nobody and the daemon only when a token is set (`api-token`), and both bind loopback for that reason. Loopback is what makes the checks here necessary rather than sufficient: a browser running on the same machine *is* a local client, so "only this machine may reach it" does not mean "only this machine's owner may drive it". Two attacks follow from that, and each guard below closes one. **Cross-site request forgery.** Any page the operator visits can `fetch` a loopback URL. `same-origin?` rejects the write whenever the browser stamps `Origin`. `edn-body?` closes the case where it does not: `application/edn` is not a CORS-*simple* content type, so a browser must preflight it, and a server answering no CORS headers fails that preflight before the request is ever sent. **DNS rebinding.** `same-origin?` compares `Origin` against the request's own `Host`, so an attacker controlling both — a domain that re-resolves to 127.0.0.1 once the page is loaded — satisfies it. `host-allowed?` is the check that does not fold, because the `Host` header must then name the interface the server was actually started on.
Synthesize a knowledge base of a chosen shape.
The two other kinds of KB are given: a shipped ontology (vaelii.host.starter) is
fixed content, and an imported corpus (vaelii.impl.io.import, or a translated one)
is whatever the source says. Neither lets you ask what happens at ten times the
rules, and that is the question a scale or behaviour measurement is made of. So this
namespace generates a KB from a handful of numbers — how many types, individuals,
predicates, facts and rules, how the rules split forward/backward, how many of them are
defeasible — and each number is a knob the browser renders as a slider (knobs).
Two properties make a generated KB usable as a measurement rather than as noise:
plan's three draw streams owns a java.util.Random
seeded from the plan seed and its own constant (stream-seeds), so the same
parameters give the same KB whichever order a reader realizes the streams in — a
shape can be reproduced from the numbers alone, and a run compared against a rerun.plan is pure — the whole KB as data, nothing asserted. load-into asserts it,
reporting progress through an optional :on-progress callback (which may throw to
cancel the load, the flag vaelii.browser.catalog cancels on).
Synthesize a knowledge base of a chosen **shape**. The two other kinds of KB are given: a shipped ontology (`vaelii.host.starter`) is fixed content, and an imported corpus (`vaelii.impl.io.import`, or a translated one) is whatever the source says. Neither lets you ask *what happens at ten times the rules*, and that is the question a scale or behaviour measurement is made of. So this namespace generates a KB from a handful of numbers — how many types, individuals, predicates, facts and rules, how the rules split forward/backward, how many of them are defeasible — and each number is a knob the browser renders as a slider (`knobs`). Two properties make a generated KB usable as a measurement rather than as noise: * **Deterministic.** Each of `plan`'s three draw streams owns a `java.util.Random` seeded from the plan seed and its own constant (`stream-seeds`), so the same parameters give the same KB whichever order a reader realizes the streams in — a shape can be reproduced from the numbers alone, and a run compared against a rerun. * **Stratified.** Predicates are split into layers: facts populate layer 0, and a rule concluding a layer-k predicate draws its antecedents only from layers below k. The rule set is therefore acyclic, so forward chaining cascades base → derived → further-derived and terminates, instead of the runaway recursion a rule set wired at random produces. Individuals and predicates are Zipf-sampled, so the corpus has hot terms and a long tail like a real one rather than a uniform smear. `plan` is pure — the whole KB as data, nothing asserted. `load-into` asserts it, reporting progress through an optional `:on-progress` callback (which may throw to cancel the load, the flag `vaelii.browser.catalog` cancels on).
The browser editor's line format: a stored sentex as the sentence its author would type back in.
The editor seeds its textarea with these sentences and diffs the text it gets back
against them by content, so a sentence spelled two ways would turn an untouched line
into a retract plus an assert of the same fact. It lives here rather than in the
browser because spelling a rule's wrappers reads the rule record's slots
(rules/rewrap-sentex), and the browser requires no vaelii.impl namespace.
The browser editor's line format: a stored sentex as the sentence its author would type back in. The editor seeds its textarea with these sentences and diffs the text it gets back against them **by content**, so a sentence spelled two ways would turn an untouched line into a retract plus an assert of the same fact. It lives here rather than in the browser because spelling a rule's wrappers reads the rule record's slots (`rules/rewrap-sentex`), and the browser requires no `vaelii.impl` namespace.
Ontology KB files: declarative content held as plain text on the classpath rather than as code.
A KB file is a list of ordinary vaelii sentences — one s-expression each, with
;; line comments and blank lines allowed — named for the context its sentences
assert into, and grouped term-centrically: every sentence about a vocabulary
term sits together, and the terms run in natural sort order. A rule is just a
sentence carrying an implies / set/*Rule / exceptWhen wrapper.
The format itself — reader and writer both — is vaelii.impl.io.text, which is
where its two non-sentence spellings live ((set/monotonic S) and (set/inert S))
and what vaelii.core/export-text! writes. What is here is the classpath side: the
shallow tree under resources/kb/ and how a layer's files are discovered in it.
The files live under resources/kb/, in two layers:
kb/upper/<domain>.txt the upper ontology, every file loaded into CxUniverse
(see vaelii.host.core-context)
kb/middle/<C>.txt theory contexts, each seeing CxUniverse
An upper file is named for its domain (core.txt is the vocabulary); a middle file's
name is its context. Only the layer a caller names is discovered, so a sibling
directory under kb/ that names no layer here is not loaded: kb/koinii/ is one, an
application's own context files, which that application loads for itself. What stays
in code (vaelii.host.core-context, vaelii.host.starter) is the order the layers
load in and the one computed batch. Sentences read with clojure.edn, so a KB file
is data and can never run code.
Ontology KB files: declarative content held as **plain text on the classpath**
rather than as code.
A KB file is a list of ordinary vaelii sentences — one s-expression each, with
`;;` line comments and blank lines allowed — named for the context its sentences
assert into, and grouped **term-centrically**: every sentence about a vocabulary
term sits together, and the terms run in natural sort order. A rule is just a
sentence carrying an `implies` / `set/*Rule` / `exceptWhen` wrapper.
**The format itself — reader and writer both — is `vaelii.impl.io.text`**, which is
where its two non-sentence spellings live (`(set/monotonic S)` and `(set/inert S)`)
and what `vaelii.core/export-text!` writes. What is here is the *classpath* side: the
shallow tree under `resources/kb/` and how a layer's files are discovered in it.
The files live under `resources/kb/`, in two layers:
kb/upper/<domain>.txt the upper ontology, every file loaded into CxUniverse
(see vaelii.host.core-context)
kb/middle/<C>.txt theory contexts, each seeing CxUniverse
An upper file is named for its domain (`core.txt` is the vocabulary); a middle file's
*name* is its context. Only the layer a caller names is discovered, so a sibling
directory under `kb/` that names no layer here is not loaded: `kb/koinii/` is one, an
application's own context files, which that application loads for itself. What stays
in **code** (vaelii.host.core-context, vaelii.host.starter) is the *order* the layers
load in and the one computed batch. Sentences read with `clojure.edn`, so a KB file
is data and can never run code.Headless EDN-over-HTTP daemon: one JVM owns one KB and serves it to remote clients
(vaelii.host.client). A thin reitit-ring + jetty layer over vaelii.core, the
network dual of the in-process API.
Wire format is EDN. A sentence is a symbol s-expression — (dog Muffet), ?x,
(genl dog animal) — which EDN round-trips losslessly; JSON would mangle the symbols.
The body of every call is {:op <keyword> :args [...]}, and the reply is
{:ok true :result …} or {:ok false :error "…"}. EDN is read with
clojure.edn/read-string (never clojure.core/read-string), so an untrusted body
cannot evaluate code — EDN has no reader-eval.
A refusal's :type is a plain keyword — :body-too-large, :not-edn,
:cross-origin, :bad-host — and so is the :type an engine ex-info carries
through. The protocol is what a client written against another build discriminates
on, so it cannot be qualified by the namespace that happens to serve it: a
::-qualified keyword names this namespace, and a client matching on it would be
matching on where the daemon's code lives.
The daemon is the single writer (docs/storage.md, the single-writer contract): it owns the one process allowed to mutate the store, so it serializes every op through one monitor. Concurrent client writes therefore apply one at a time and cannot interleave; reads pay the same lock, which is conservative but keeps the contract simple.
Only the allowlisted ops are reachable (ops). Each is a vaelii.core fn with
the KB supplied by the daemon — the client sends only the op and the remaining args —
so no client can reach an arbitrary var. Sentex records in a result are projected to
plain maps before they hit the wire (the sentex-map contract), so the client reads
them back without the impl record class.
The change feed is the one thing that is not a vaelii.core fn (feed-ops), and
it is a table of its own for that reason: core/watch takes a callback, so what a
remote caller holds open instead is a subscription with a cursor — :watch,
:poll, :unwatch, :watchers, over the per-handler registry app builds
(vaelii.host.subscribe, docs/feed.md). A :poll that waits runs outside the
monitor; everything else about them is an ordinary EDN op.
One shared bearer token authenticates the caller. With VAELII_API_TOKEN set
(guard/api-token), every request presents Authorization: Bearer <token> or is
answered 401 with a WWW-Authenticate: Bearer challenge; GET /health is the one
route that answers without it. One token for the process, not a session and not an
identity — per-caller identity is a reverse proxy's job, and this is the check that
has to exist below it. Binding anything but loopback requires a token (-main
refuses to start otherwise); on the loopback default it is optional, and a daemon
without one is drivable by every process on the machine.
vaelii.host.guard covers what a token does not, and matters most on the open
loopback daemon: POST /op requires Content-Type: application/edn, refuses a
cross-origin Origin, and answers only to a Host naming the interface it was
started on. Together those stop a page the operator happens to visit from driving
the KB over loopback — which binding to loopback alone does not.
Headless EDN-over-HTTP daemon: one JVM owns one KB and serves it to remote clients
(`vaelii.host.client`). A thin reitit-ring + jetty layer over `vaelii.core`, the
network dual of the in-process API.
**Wire format is EDN.** A sentence is a symbol s-expression — `(dog Muffet)`, `?x`,
`(genl dog animal)` — which EDN round-trips losslessly; JSON would mangle the symbols.
The body of every call is `{:op <keyword> :args [...]}`, and the reply is
`{:ok true :result …}` or `{:ok false :error "…"}`. EDN is read with
`clojure.edn/read-string` (never `clojure.core/read-string`), so an untrusted body
cannot evaluate code — EDN has no reader-eval.
**A refusal's `:type` is a plain keyword** — `:body-too-large`, `:not-edn`,
`:cross-origin`, `:bad-host` — and so is the `:type` an engine `ex-info` carries
through. The protocol is what a client written against another build discriminates
on, so it cannot be qualified by the namespace that happens to serve it: a
`::`-qualified keyword names *this* namespace, and a client matching on it would be
matching on where the daemon's code lives.
**The daemon is the single writer** (docs/storage.md, the single-writer contract): it
owns the one process allowed to mutate the store, so it serializes every op through
one monitor. Concurrent client writes therefore apply one at a time and cannot
interleave; reads pay the same lock, which is conservative but keeps the contract
simple.
**Only the allowlisted ops are reachable** (`ops`). Each is a `vaelii.core` fn with
the KB supplied by the daemon — the client sends only the op and the remaining args —
so no client can reach an arbitrary var. Sentex records in a result are projected to
plain maps before they hit the wire (the `sentex`-map contract), so the client reads
them back without the `impl` record class.
**The change feed is the one thing that is not a `vaelii.core` fn** (`feed-ops`), and
it is a table of its own for that reason: `core/watch` takes a callback, so what a
remote caller holds open instead is a subscription with a **cursor** — `:watch`,
`:poll`, `:unwatch`, `:watchers`, over the per-handler registry `app` builds
(`vaelii.host.subscribe`, docs/feed.md). A `:poll` that waits runs **outside** the
monitor; everything else about them is an ordinary EDN op.
**One shared bearer token authenticates the caller.** With `VAELII_API_TOKEN` set
(`guard/api-token`), every request presents `Authorization: Bearer <token>` or is
answered 401 with a `WWW-Authenticate: Bearer` challenge; `GET /health` is the one
route that answers without it. One token for the process, not a session and not an
identity — per-caller identity is a reverse proxy's job, and this is the check that
has to exist below it. Binding anything but loopback **requires** a token (`-main`
refuses to start otherwise); on the loopback default it is optional, and a daemon
without one is drivable by every process on the machine.
`vaelii.host.guard` covers what a token does not, and matters most on the open
loopback daemon: `POST /op` requires `Content-Type: application/edn`, refuses a
cross-origin `Origin`, and answers only to a `Host` naming the interface it was
started on. Together those stop a page the operator happens to visit from driving
the KB over loopback — which binding to loopback alone does not.A starter common-sense KB: a documented, schema-only upper + middle ontology. It loads the upper ontology into CxUniverse (vaelii.host.core-context), then the middle contexts, each a KB file on the classpath under resources/kb/:
kb/upper/, every file in CxUniverse): what things are, always true, like
genl. Split by domain, one file each:
kb/middle/, one context per file, between CxUniverse and CxWell): how the
definitional things interrelate, where several overlapping theories can coexist.
The topology, most general (top) to most specific (bottom): CxUniverse holds the upper
ontology; each middle context sees CxUniverse and not the other middle contexts; and
CxWell sees every middle context but the opt-in ones. Each middle file wires itself in
with its own genlCx edges, so the topology is data. No cast and no contingent facts
ship: the starter is a schema, and
contingent data (a cast, worked examples, the Aesop fables) belongs below CxWell
and lives in the tests that need it.
The unit table is the one place individuals ship, and it applies that rule rather than excepting itself from it: a minute is sixty seconds by stipulation, so the factor is vocabulary and not a measurement anybody took. measure.txt states the test it holds a unit to.
What stays in code here is the order the layers load in — loading order is logic, the definitional layer must precede the theories that reason over it — and the one computed batch (every type is also a unary_predicate). Within a layer, every file present is loaded (discovered from the classpath), so adding a domain or a theory is dropping a file in kb/upper/ or kb/middle/, no code change.
A starter common-sense KB: a documented, **schema-only** upper + middle ontology.
It loads the upper ontology into CxUniverse (vaelii.host.core-context), then the
middle contexts, each a KB file on the classpath under resources/kb/:
* upper (`kb/upper/`, every file in CxUniverse): what things *are*, always true, like
`genl`. Split by domain, one file each:
- core.txt — the vocabulary the engine interprets and the top of the
ontology.
- abstract.txt — the abstract type skeleton (physical/intangible and
their kinds) + the structural relations partOf/locatedIn.
- organism.txt — the biological taxonomy + its disjointness.
- life.txt — the organism relations (parentOf, siblingOf, flies,
mortal, birthYearOf, olderThan, …) with arg + metadata.
- society.txt — the social relations (marriedTo, likes, owns).
- measure.txt — the theory of measurement: the two measure terms, the
dimensionOf/conversionFactor table with the units that
fill it, the comparisons, weightOf / heightOf, and the
sign vocabulary for the quantities nobody has a figure
for (signOf / trendOf / the qualitative* arithmetic).
- reflection.txt — the expression lattice (atomic and non-atomic, open and
closed, well-formed and ill-formed expressions) and the
use/mention vocabulary (proposition, means, denotes,
expresses).
- space.txt — qualitative space, four independent calculi: RCC-8
topology (eight base + six derived), cardinal direction
(nine + four), relative direction over a frame's own axes
(nine + four, the frame being the context), and
qualitative distance (seven + three).
- time.txt — qualitative time: Allen's interval relations (thirteen
base + seven derived), the point algebra over instants,
the three calendar constructors and the InstantFn moment
a calendar term's startOf and endOf are computed as, plus
the length / totalDuration / overlapDuration vocabulary
the arithmetic computes over.
* middle (`kb/middle/`, one context per file, between CxUniverse and CxWell): how the
definitional things *interrelate*, where several overlapping theories can coexist.
- CxAnatomy.txt — what kinds of thing have what kinds of part.
- CxBiology.txt — birds fly by default except penguins; organisms
are mortal; flight enables travel; sleep is what the
theory is willing to assume.
- CxChange.txt — a simple event calculus: a state persists until an
event ends it, so holdsAt is inertia over what
initiates and terminates say.
- CxComputing.txt — relations over software tools, their invocations and
receipts, media resources and DNS names, and
the kinds the relations are typed over.
Opt-in: it sees CxUniverse and CxWell does not
see it.
- CxKinship.txt — grandparentOf, ancestorOf, olderThan, and parenthood
from maternity and paternity.
- CxMereology.txt — a part is located where its whole is; owning a whole
entails owning its parts.
- CxNormalPhysicalConditions.txt — the states of matter at room temperature
and pressure: stone, wood and glass are solid,
mercury is liquid, a metal is solid by default.
Opt-in: it sees CxUniverse and CxWell does not
see it.
- CxPerception.txt — perception relations: perceives, sees, seeImage,
watchVideo. Opt-in: it sees CxUniverse and CxWell
does not see it.
- CxSize.txt — comparative size: stated between kinds, computed
between objects from their measures.
- CxSocial.txt — what acquaintance follows from; employment as one way
of belonging; the general relationship and
dwelling vocabulary over two persons.
- CxSocialExtension.txt — relationship and plural-identity vocabulary
narrower than CxSocial's own. Opt-in: it sees
CxSocial and CxWell does not see it. The
Extension suffix names a theory that extends
an existing starter theory with vocabulary
most contexts under the base theory have no
occasion to see.
The topology, most general (top) to most specific (bottom): CxUniverse holds the upper
ontology; each middle context sees CxUniverse and not the other middle contexts; and
CxWell sees every middle context but the opt-in ones. Each middle file wires itself in
with its own `genlCx` edges, so the topology is data. **No cast and no contingent facts
ship**: the starter is a schema, and
contingent data (a cast, worked examples, the Aesop fables) belongs below CxWell
and lives in the tests that need it.
The unit table is the one place individuals ship, and it applies that rule rather
than excepting itself from it: a minute is sixty seconds by stipulation, so the
factor is vocabulary and not a measurement anybody took. measure.txt states
the test it holds a unit to.
What stays in code here is the *order the layers* load in — loading order is logic,
the definitional layer must precede the theories that reason over it — and the one
computed batch (every type is also a unary_predicate). Within a layer, every file
present is loaded (discovered from the classpath), so adding a domain or a theory is
dropping a file in kb/upper/ or kb/middle/, no code change.The change feed with a cursor where the in-process one has a callback — the daemon-side state a remote caller holds a feed open against.
core/watch takes a function, and a function does not cross an EDN wire (the same
wall :export's :on-progress hits). So the wire's half of the feed is not the
callback marshalled somehow; it is the one thing a request/response protocol can
carry, which is state with a cursor: the daemon registers an ordinary listener of
its own, that listener files each event into a bounded ring, and a caller reads the
ring forward from where it left off. Three ops — register, read, drop — every one of
them EDN in and EDN out, so the guards, the client and the error taxonomy that already
exist carry it unchanged (docs/operations.md).
A cursor counts events, not handles. It starts at 0 when the subscription is
registered and advances by one per delivered event, so a caller compares nothing and
stores one integer. poll answers the events past the cursor it was handed and the
cursor to send next time.
The ring is bounded, and falling off it is said out loud. A subscriber that stops
reading must not grow the daemon's heap, so the ring keeps max-events and drops the
oldest past it — and the count of what it dropped is reported as :lagged on the
next poll. That number is the whole reason this is usable: a feed with a silent gap
is strictly worse than polling, because the caller believes it is current and is not.
:lagged is present on every reply, zero and all, so a client that forgets to read it
is a client that cannot have one.
A token that names no subscription is refused, never answered empty. The same
argument: a reaped, dropped or invented token answering {:events []} is a feed that
has silently stopped. :unknown-subscription says so.
What a subscription costs the daemon, and what bounds it. One listener on the
KB's feed and one ring of at most max-events events; max-subscriptions of those at
once, and one that nobody has polled inside idle-ms is reaped at the next call.
Nothing here authenticates the caller — that is the bearer token's job, one layer out
(vaelii.host.serve) — but heap a stranger can allocate wants a ceiling whether or
not it is authenticated, and the reap is what keeps an abandoned subscription from
holding a slot against a live one.
The wait happens here, outside the daemon's monitor. A long poll parks on the
subscription's own signal object, so a writer serialized behind serve's one monitor
runs to completion while a poll is parked — the feature is about liveness, and a
parked poll that blocked every writer would be a global stall wearing its name. The
writing thread's only cost is the swap that files the event and a notifyAll on a
monitor no poller holds for longer than a compare.
The three entry points are spelled without ! for core/watch's reason: nothing here
destroys stored knowledge (docs/api.md). See docs/feed.md, "Across the wire".
The change feed with a **cursor** where the in-process one has a callback — the
daemon-side state a remote caller holds a feed open against.
`core/watch` takes a function, and a function does not cross an EDN wire (the same
wall `:export`'s `:on-progress` hits). So the wire's half of the feed is not the
callback marshalled somehow; it is the one thing a request/response protocol can
carry, which is **state with a cursor**: the daemon registers an ordinary listener of
its own, that listener files each event into a bounded ring, and a caller reads the
ring forward from where it left off. Three ops — register, read, drop — every one of
them EDN in and EDN out, so the guards, the client and the error taxonomy that already
exist carry it unchanged (docs/operations.md).
**A cursor counts events, not handles.** It starts at 0 when the subscription is
registered and advances by one per delivered event, so a caller compares nothing and
stores one integer. `poll` answers the events past the cursor it was handed and the
cursor to send next time.
**The ring is bounded, and falling off it is said out loud.** A subscriber that stops
reading must not grow the daemon's heap, so the ring keeps `max-events` and drops the
oldest past it — and the *count* of what it dropped is reported as `:lagged` on the
next poll. That number is the whole reason this is usable: a feed with a silent gap
is strictly worse than polling, because the caller believes it is current and is not.
`:lagged` is present on every reply, zero and all, so a client that forgets to read it
is a client that cannot have one.
**A token that names no subscription is refused, never answered empty.** The same
argument: a reaped, dropped or invented token answering `{:events []}` is a feed that
has silently stopped. `:unknown-subscription` says so.
**What a subscription costs the daemon, and what bounds it.** One listener on the
KB's feed and one ring of at most `max-events` events; `max-subscriptions` of those at
once, and one that nobody has polled inside `idle-ms` is reaped at the next call.
Nothing here authenticates the caller — that is the bearer token's job, one layer out
(`vaelii.host.serve`) — but heap a stranger can allocate wants a ceiling whether or
not it is authenticated, and the reap is what keeps an abandoned subscription from
holding a slot against a live one.
**The wait happens here, outside the daemon's monitor.** A long poll parks on the
subscription's own signal object, so a writer serialized behind `serve`'s one monitor
runs to completion while a poll is parked — the feature is about liveness, and a
parked poll that blocked every writer would be a global stall wearing its name. The
writing thread's only cost is the swap that files the event and a `notifyAll` on a
monitor no poller holds for longer than a compare.
The three entry points are spelled without `!` for `core/watch`'s reason: nothing here
destroys stored knowledge (docs/api.md). See docs/feed.md, "Across the wire".Bring a KB's shipped contexts up to the running engine's.
A KB stores the starter ontology it was built with, and the engine that opens it later
ships its own: a strength marked set/monotonic, a vocabulary term added to core.txt, a
disjointness the ontology stopped stating. sync! makes the KB state what this engine
ships, context by context, and leaves everything else alone.
What is shipped is what starter/load-into produces, read off a scratch in-memory KB
it is loaded into — not the text of the files. The two differ: genlCx edges are stored
in CxUniverse whichever file states them, and the closing unary_predicate batch is
stated by no file. Each premise is attributed to the load that stored it: CxUniverse
for the upper ontology, the middle context for a middle file.
Which contexts are synced exactly: CxUniverse and the kb/middle/ contexts. A
premise there that the engine does not ship is retracted, a strength that differs is
restated, and a missing one is asserted. CxUniverse is locked to the files with two
exceptions, which sync neither retracts nor restates: a genlCx premise, stored in
CxUniverse whichever file or author states it, and a sentex that is not a premise (a
lifted decontextualized_predicate copy, a rule firing). An author's own context, wired
below CxUniverse, is not one of them and is not read.
Only added to: any other context a shipped file's content is stored in.
Only the layers a KB has. A middle file whose context holds nothing in the KB is
not loaded into it. The upper ontology is loaded only into a KB whose CxUniverse holds
a premise the upper load ships, and the unary_predicate batch only into one that also
holds a mark the batch ships.
Comparison is by the form a KB file writes (text/premise-entries): a fact or rule
under its strength wrapper, an exceptWhen as the wrapper it was asserted as. A
strength the engine raised is only an assertion, since assert raises a held
premise's strength in place; one it lowered is the form retracted and the weaker one
asserted, since nothing lowers a strength in place.
One batch. The sync is one v/edit!: the additions first, then the retractions,
one settle. So the belief a retraction would sweep and the addition rebuild keeps its
witness through the batch, where retracting first tore the TMS down only to build it
back up. What cannot go in that order — a retraction of a record an addition lands on —
goes before it (sync!).
Write-back is the other direction: write-back! edits the shipped files to state
what a KB's exact contexts gained and lost since a held reading, one line per
premise. The development browser runs it for the starter (docs/catalog.md).
Bring a KB's shipped contexts up to the running engine's. A KB stores the starter ontology it was built with, and the engine that opens it later ships its own: a strength marked `set/monotonic`, a vocabulary term added to core.txt, a disjointness the ontology stopped stating. `sync!` makes the KB state what this engine ships, context by context, and leaves everything else alone. **What is shipped** is what `starter/load-into` produces, read off a scratch in-memory KB it is loaded into — not the text of the files. The two differ: `genlCx` edges are stored in CxUniverse whichever file states them, and the closing `unary_predicate` batch is stated by no file. Each premise is attributed to the load that stored it: CxUniverse for the upper ontology, the middle context for a middle file. **Which contexts are synced exactly**: CxUniverse and the `kb/middle/` contexts. A premise there that the engine does not ship is retracted, a strength that differs is restated, and a missing one is asserted. CxUniverse is locked to the files with two exceptions, which sync neither retracts nor restates: a `genlCx` premise, stored in CxUniverse whichever file or author states it, and a sentex that is not a premise (a lifted `decontextualized_predicate` copy, a rule firing). An author's own context, wired below CxUniverse, is not one of them and is not read. **Only added to**: any other context a shipped file's content is stored in. **Only the layers a KB has.** A middle file whose context holds nothing in the KB is not loaded into it. The upper ontology is loaded only into a KB whose CxUniverse holds a premise the upper load ships, and the `unary_predicate` batch only into one that also holds a mark the batch ships. Comparison is by the form a KB file writes (`text/premise-entries`): a fact or rule under its strength wrapper, an `exceptWhen` as the wrapper it was asserted as. A strength the engine **raised** is only an assertion, since `assert` raises a held premise's strength in place; one it **lowered** is the form retracted and the weaker one asserted, since nothing lowers a strength in place. **One batch.** The sync is one `v/edit!`: the additions first, then the retractions, one settle. So the belief a retraction would sweep and the addition rebuild keeps its witness through the batch, where retracting first tore the TMS down only to build it back up. What cannot go in that order — a retraction of a record an addition lands on — goes before it (`sync!`). **Write-back** is the other direction: `write-back!` edits the shipped files to state what a KB's exact contexts gained and lost since a `held` reading, one line per premise. The development browser runs it for the starter (docs/catalog.md).
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |