What knowledge bases this process can load, and the lifecycle of loading one.
Everything above the engine assumes it is holding the KB. A browser that lists the KBs available, loads one while you watch, and switches to it needs two things the engine does not have: a description of a KB that has not been loaded yet, and somewhere for a load that takes minutes to run while the pages keep answering. Those are the two halves here.
A source is a KB you could load, as data — a kind, a name, and wherever the content comes from. Six kinds:
| kind | content | loader |
|---|---|---|
:core | the upper ontology alone, in CxUniverse | vaelii.host.core-context |
:starter | the shipped schema-only ontology | vaelii.host.starter |
:generated | synthesized from numbers — types, rules, a fwd mix | vaelii.host.io.generate |
:corpus | a translated sentence corpus (OpenCyc) | a foreign reader, :cyc-corpus |
:dump | a vaelii export dump | vaelii.impl.io.import |
:store | an on-disk KB already in vaelii's own format | opened in place |
The first three ship in this repo and are always offered. The last three are found:
each directory on the search path (VAELII_KB_PATH, else ./kbs and ~/.vaelii/kbs)
is probed, and what marks it — a corpus meta.edn, a dump meta.edn, a records/ +
index/ pair — decides its kind. A catalog.edn (VAELII_KB_CATALOG, else
~/.vaelii/catalog.edn) names sources outside the search path. Nothing about a
machine's paths is baked into the repo.
An entry is a source that has been loaded, or is loading: a KB, a status, and a
progress reading the loaders report into (:on-progress, reported by every loader —
the corpus reader, io.import/import-dump and io.generate/load-into). The running
half of that is not here: a load is a job (vaelii.browser.jobs), which is what gives
it a thread of its own, the progress reading, the cancel flag and the report — so an
entry carries the job's id and reads its status rather than keeping one. One load runs
at a time, since a load claims this process's writer, and cancelling one is cooperative:
the loaders have no other safe interruption point, and an import is not a transaction,
so a cancelled load leaves the KB holding what had already landed.
A KB is readable before it is finished. activate asks only that an entry hold a
KB, so the one arriving can be the one every page reads — a corpus is browsable from
its first thousand sentexes, and a store that opens in seconds is browsable while
belief is still being rebuilt behind it. What that costs a reader is completeness, not
correctness, and active-caveat is what says so.
And a KB can go back out. export-entry! writes a loaded one as an export dump as
a job like any other, which closes the loop: a dump written under the search path is a
:dump source the moment its meta.edn lands, so exporting and reloading needs nothing
outside this namespace.
Unloading never deletes an on-disk KB. A memory-backed entry has its stores cleared (they would otherwise hold the corpus for the life of the JVM); every other backend is closed — the file lock released, the directory left exactly as it was, an adapter's connection closed. The same directory can then be loaded again, or opened by another process.
What knowledge bases this process can load, and the lifecycle of loading one. Everything above the engine assumes it is holding *the* KB. A browser that lists the KBs available, loads one while you watch, and switches to it needs two things the engine does not have: a description of a KB that has not been loaded yet, and somewhere for a load that takes minutes to run while the pages keep answering. Those are the two halves here. **A source** is a KB you could load, as data — a kind, a name, and wherever the content comes from. Six kinds: | kind | content | loader | |--------------|-----------------------------------------------------|--------| | `:core` | the upper ontology alone, in CxUniverse | `vaelii.host.core-context` | | `:starter` | the shipped schema-only ontology | `vaelii.host.starter` | | `:generated` | synthesized from numbers — types, rules, a fwd mix | `vaelii.host.io.generate` | | `:corpus` | a translated sentence corpus (OpenCyc) | a foreign reader, `:cyc-corpus` | | `:dump` | a vaelii export dump | `vaelii.impl.io.import` | | `:store` | an on-disk KB already in vaelii's own format | opened in place | The first three ship in this repo and are always offered. The last three are **found**: each directory on the search path (`VAELII_KB_PATH`, else `./kbs` and `~/.vaelii/kbs`) is probed, and what marks it — a corpus `meta.edn`, a dump `meta.edn`, a `records/` + `index/` pair — decides its kind. A `catalog.edn` (`VAELII_KB_CATALOG`, else `~/.vaelii/catalog.edn`) names sources outside the search path. Nothing about a machine's paths is baked into the repo. **An entry** is a source that has been loaded, or is loading: a KB, a status, and a progress reading the loaders report into (`:on-progress`, reported by every loader — the corpus reader, `io.import/import-dump` and `io.generate/load-into`). The running half of that is not here: a load is a **job** (`vaelii.browser.jobs`), which is what gives it a thread of its own, the progress reading, the cancel flag and the report — so an entry carries the job's id and reads its status rather than keeping one. One load runs at a time, since a load claims this process's writer, and cancelling one is cooperative: the loaders have no other safe interruption point, and an import is not a transaction, so a cancelled load leaves the KB holding what had already landed. **A KB is readable before it is finished.** `activate` asks only that an entry hold a KB, so the one arriving can be the one every page reads — a corpus is browsable from its first thousand sentexes, and a store that opens in seconds is browsable while belief is still being rebuilt behind it. What that costs a reader is completeness, not correctness, and `active-caveat` is what says so. **And a KB can go back out.** `export-entry!` writes a loaded one as an export dump as a job like any other, which closes the loop: a dump written under the search path is a `:dump` source the moment its `meta.edn` lands, so exporting and reloading needs nothing outside this namespace. **Unloading never deletes an on-disk KB.** A memory-backed entry has its stores cleared (they would otherwise hold the corpus for the life of the JVM); every other backend is *closed* — the file lock released, the directory left exactly as it was, an adapter's connection closed. The same directory can then be loaded again, or opened by another process.
(activate key)Make entry key the one the browser reads. Anything holding a KB can be activated,
a load still running included; only an entry with no KB yet is refused. A half-loaded KB
answers about a prefix, the open-world condition, and reading beside the loader is
sound for the single-writer reason (docs/storage.md). A reader is owed being told
(active-caveat), not being refused. The entry is tested inside the swap! that
activates it, so a concurrent unload! cannot leave :active naming a dropped key.
Make entry `key` the one the browser reads. Anything **holding a KB** can be activated, a load still running included; only an entry with no KB yet is refused. A half-loaded KB answers about a *prefix*, the open-world condition, and reading beside the loader is sound for the single-writer reason (docs/storage.md). A reader is owed being *told* (`active-caveat`), not being refused. The entry is tested inside the `swap!` that activates it, so a concurrent `unload!` cannot leave `:active` naming a dropped key.
(active)The key of the active entry, or nil.
The key of the active entry, or nil.
(active-caveat)What is provisional about the KB the browser is reading, or nil when nothing is.
{:key :name :status :progress :belief? :recoverable? :rebuilding? :rebuild}, :rebuild
being v/rebuild-progress while belief is rebuilt behind an installed image. Two
independent reasons an answer can be less than the whole truth:
recover builds both, :belief? false
skips both): every believed answer is empty and a fully stored KB renders with no
types or contexts.The second outlives the first (a store opened without :recover? is :done and stays
so), which is why it is reported beside the status. The TMS is what is probed. An empty
KB is empty, not beliefless; an entry that is not an in-process KB has nothing to
report.
What is provisional about the KB the browser is reading, or nil when nothing is.
`{:key :name :status :progress :belief? :recoverable? :rebuilding? :rebuild}`, `:rebuild`
being `v/rebuild-progress` while belief is rebuilt behind an installed image. Two
independent reasons an answer can be less than the whole truth:
- the load is **still running** (or stopped part-way), so what is stored is a prefix;
- **belief and the taxonomy are not built** (`recover` builds both, `:belief? false`
skips both): every believed answer is empty and a fully stored KB renders with no
types or contexts.
The second outlives the first (a store opened without `:recover?` is `:done` and stays
so), which is why it is reported beside the status. The TMS is what is probed. An empty
KB is empty, not beliefless; an entry that is not an in-process KB has nothing to
report.(active-kb)The KB the browser should be reading, or nil when nothing is loaded.
The KB the browser should be reading, or nil when nothing is loaded.
The sources that need no files beyond the ones in this repo. :options is what the
UI offers per source, as data: each entry is a form control, and load-source reads
the same keys back out of the params map.
The sources that need no files beyond the ones in this repo. `:options` is what the UI offers per source, as data: each entry is a form control, and `load-source` reads the same keys back out of the params map.
(cancel! key)Ask a running load to stop at its next progress report. ! because what it leaves
behind is a half-loaded KB — the loaders write as they go and none of them is a
transaction.
Ask a running load to stop at its next progress report. `!` because what it leaves behind is a half-loaded KB — the loaders write as they go and none of them is a transaction.
(cancel-export!)Ask the export that is running to stop at its next chunk boundary, and answer
whether there was one to ask. ! because what it leaves behind is a directory holding
part of a dump.
Asked of the running set rather than of jobs/latest: the panel shows the last export's
report for an hour after it settles, so the newest export of any status is routinely one
that finished this morning, and this reports on the dump that is still being written
rather than on whichever one the panel happens to be showing.
Ask the export that is **running** to stop at its next chunk boundary, and answer whether there was one to ask. `!` because what it leaves behind is a directory holding part of a dump. Asked of the running set rather than of `jobs/latest`: the panel shows the last export's report for an hour after it settles, so the newest export of any status is routinely one that finished this morning, and this reports on the dump that is still being written rather than on whichever one the panel happens to be showing.
(claim-space!)The next free :space number.
The next free `:space` number.
(classify d)The kind of KB in directory d, or nil. Reads the marker each writer leaves: a corpus's
meta.edn carries the context order, a dump's carries a :format-version, and a vaelii
store is a records/ directory with its own format.edn. The records half is the
whole marker: only :disk-log keeps a durable index/, and :disk-columnar,
:disk-dense and :disk-memory write none. A KB whose records are on a server
(:pg-disk-log, :pg-snapshot) has no records/ here; its marker is
index/records.edn, which open-kb reopens it from.
The kind of KB in directory `d`, or nil. Reads the marker each writer leaves: a corpus's `meta.edn` carries the context order, a dump's carries a `:format-version`, and a vaelii store is a `records/` directory with its own `format.edn`. **The records half is the whole marker**: only `:disk-log` keeps a durable `index/`, and `:disk-columnar`, `:disk-dense` and `:disk-memory` write none. A KB whose records are on a server (`:pg-disk-log`, `:pg-snapshot`) has no `records/` here; its marker is `index/records.edn`, which `open-kb` reopens it from.
(entries)Every entry, in load order, as public entries.
Every entry, in load order, as public entries.
(entry key)One entry, whole (:kb included) — for a caller that wants the KB itself.
One entry, whole (`:kb` included) — for a caller that wants the KB itself.
(entry-of kb)The entry holding kb, or nil for a KB this registry never heard of. By identity,
like exporting-kb? and write-blocked?.
The entry holding `kb`, or nil for a KB this registry never heard of. By identity, like `exporting-kb?` and `write-blocked?`.
(export-entry! key dir opts)Write the KB in entry key out as a dump in dir, on its own thread, and return the
job. opts are vaelii.core/export!'s (:variant, :compression), plus :run-in, a
wrapper the walk runs inside (the browser's write monitor, so a synchronous write already
past the refusals drains before the walk starts). Three refusals:
export! walks record by record with no snapshot.While the walk runs exporting-kb? answers true for this KB, and the browser's write
entry point refuses with it, since the job claims no writer (a daemon's export is filed
by serve under its own monitor). A blank dir writes under write-root, in a
directory server-dir names after the entry key. ! for what a cancelled one leaves:
a directory holding part of a dump, not loadable since meta.edn is written last.
Write the KB in entry `key` out as a dump in `dir`, on its own thread, and return the job. `opts` are `vaelii.core/export!`'s (`:variant`, `:compression`), plus `:run-in`, a wrapper the walk runs inside (the browser's write monitor, so a synchronous write already past the refusals drains before the walk starts). Three refusals: - **another export is running**: one at a time. A load does not refuse an export. - **the entry is not an in-process KB**: a daemon's export belongs to that daemon. - **the KB is still loading**: `export!` walks record by record with no snapshot. While the walk runs `exporting-kb?` answers true for this KB, and the browser's write entry point refuses with it, since the job claims no writer (a daemon's export is filed by `serve` under its own monitor). A blank `dir` writes under `write-root`, in a directory `server-dir` names after the entry key. `!` for what a cancelled one leaves: a directory holding part of a dump, not loadable since `meta.edn` is written last.
(exporting-kb? kb)Is kb the one a running export is still walking? Asked by identity, as
write-blocked? is, and its reciprocal: export-entry! refuses to start while a loader
writes the KB, and this lets the write entry points refuse while the walk runs, since the
walk fetches record by record with no snapshot. unload! reads it too.
Is `kb` the one a running export is still walking? Asked by identity, as `write-blocked?` is, and its reciprocal: `export-entry!` refuses to *start* while a loader writes the KB, and this lets the write entry points refuse while the walk runs, since the walk fetches record by record with no snapshot. `unload!` reads it too.
(exporting?)Is an export running? One at a time, as with loads.
Is an export running? One at a time, as with loads.
(footprint key)An estimate of what the KB in entry key costs in RAM, by component:
{:sentexes :index :records :tms :total :estimated? true} (bytes), or nil for an entry
with no in-process KB (an attached daemon holds its corpus in another process). Nothing
is measured (resident-bytes-per-sentex), and the sentex count is the trie's root count,
O(1). The belief check is not free: a {:tms :dense} KB answers -datums by draining
its node bitmap into a vector of boxed Longs, an allocation per stored sentex per render.
Two adjustments: a :disk-log KB drops the record term (its bounded hot-record LRU does
not grow with the corpus), and a KB loaded without belief drops the TMS term.
An estimate of what the KB in entry `key` costs in RAM, by component:
`{:sentexes :index :records :tms :total :estimated? true}` (bytes), or nil for an entry
with no in-process KB (an attached daemon holds its corpus in another process). Nothing
is measured (`resident-bytes-per-sentex`), and the sentex count is the trie's root count,
O(1). The belief check is not free: a `{:tms :dense}` KB answers `-datums` by draining
its node bitmap into a vector of boxed Longs, an allocation per stored sentex per render.
Two adjustments: a `:disk-log` KB drops the record term (its bounded hot-record LRU does
not grow with the corpus), and a KB loaded without belief drops the TMS term.(heap)What this JVM's heap is doing right now — {:used :committed :max} in bytes, read
off the memory MX bean.
A measurement, not an estimate, but a coarse one: :used includes garbage that has not
been collected, so it drifts up between collections and drops without anything being
freed. Read it as the structure of the curve, not as a number to subtract KBs from.
What this JVM's heap is doing **right now** — `{:used :committed :max}` in bytes, read
off the memory MX bean.
A measurement, not an estimate, but a coarse one: `:used` includes garbage that has not
been collected, so it drifts up between collections and drops without anything being
freed. Read it as the structure of the curve, not as a number to subtract KBs from.(held-premises key)sync/held of entry key's KB, or nil unless that KB is in this process and
accepts writes.
`sync/held` of entry `key`'s KB, or nil unless that KB is in this process and accepts writes.
(holder fallback)A deref-able that always yields the KB to read — the active entry's, or fallback
when nothing is loaded. This is what the browser is built against: app holds one of
these instead of a KB, so activating another entry re-points every page at once.
A deref-able that always yields the KB to read — the active entry's, or `fallback` when nothing is loaded. This is what the browser is built against: `app` holds one of these instead of a KB, so activating another entry re-points every page at once.
(human-bytes n)A byte count as something to read at a glance. Public because the browser shows byte counts of its own (a KB's estimated footprint, the heap) and there is no reason for two spellings of the same figure.
A byte count as something to read at a glance. Public because the browser shows byte counts of its own (a KB's estimated footprint, the heap) and there is no reason for two spellings of the same figure.
(in-process? key)Does entry key hold a KB in this JVM? A KB is a map with a record store in it; an
attached daemon is registered as an entry too, and its KB is somewhere else — which is
why stats and footprint have nothing to say about one, and why an export of one is
written on that daemon's host rather than here.
Does entry `key` hold a KB in *this* JVM? A KB is a map with a record store in it; an attached daemon is registered as an entry too, and its KB is somewhere else — which is why `stats` and `footprint` have nothing to say about one, and why an export of one is written on that daemon's host rather than here.
(install-memory-guard!)Attach the cache memory-pressure guard, feeding it this catalog's live KBs so the trim
reaches their per-KB caches (vaelii.core/install-memory-guard!). The browser
calls this at startup; the guard holds no list of open KBs of its own, since the engine
does not.
Attach the cache memory-pressure guard, feeding it this catalog's live KBs so the trim reaches their per-KB caches (`vaelii.core/install-memory-guard!`). The browser calls this at startup; the guard holds no list of open KBs of its own, since the engine does not.
(live-kbs)Every KB open in this JVM — the records whose per-KB caches a process-wide sweep
reaches. An attached daemon's entry holds a remote handle with no local caches, so
:records tells the two apart, the way in-process? does for one key.
Every KB open in *this* JVM — the records whose per-KB caches a process-wide sweep reaches. An attached daemon's entry holds a remote handle with no local caches, so `:records` tells the two apart, the way `in-process?` does for one key.
(load-dir path params)Start loading the KB directory at path under params, as a job, and make it the
active KB when the load finishes. Returns the entry key. The search path and the
catalog file need not list path: the directory is classified as discovery classifies
one, so params are that kind's options ({:recover? true} for a store). Throws
:unknown-source when path holds no KB, and otherwise what load-source throws.
Start loading the KB directory at `path` under `params`, as a job, and make it the
active KB when the load finishes. Returns the entry key. The search path and the
catalog file need not list `path`: the directory is classified as discovery classifies
one, so `params` are that kind's options (`{:recover? true}` for a store). Throws
`:unknown-source` when `path` holds no KB, and otherwise what `load-source` throws.(load-source source-id)(load-source source-id params)Start loading the source with id source-id under params, as a job. Returns the
entry key, or throws when the id names no source, the source is already loaded, or
another job holds this process's writer.
The entry is registered :running before this returns, so the caller can render it
immediately; entries then reports the job's progress until it settles into :done
(the KB is queryable, and activated when nothing else is) or :failed / :cancelled.
Start loading the source with id `source-id` under `params`, as a job. Returns the entry key, or throws when the id names no source, the source is already loaded, or another job holds this process's writer. The entry is registered `:running` before this returns, so the caller can render it immediately; `entries` then reports the job's progress until it settles into `:done` (the KB is queryable, and activated when nothing else is) or `:failed` / `:cancelled`.
(loading?)Is a load running? One runs at a time: a load claims this process's writer, and two at once would make each other's timings meaningless. Asked of the registry, since that is where a running load lives.
Is a load running? One runs at a time: a load claims this process's writer, and two at once would make each other's timings meaningless. Asked of the registry, since that is where a running load lives.
How many directories one search-path entry is probed for, at most. Every candidate costs
a classify (a meta.edn read) and, for a :store, a size estimate on every /kbs
request, since sources is recomputed per call. Nothing else in the browser renders an
uncapped list (docs/web.md, "Long lists continue"). The cap bounds the cost and does
not curate.
How many directories one search-path entry is probed for, at most. Every candidate costs a `classify` (a `meta.edn` read) and, for a `:store`, a size estimate on **every** `/kbs` request, since `sources` is recomputed per call. Nothing else in the browser renders an uncapped list (docs/web.md, "Long lists continue"). The cap bounds the cost and does not curate.
(memory)The memory picture the browser shows: the JVM heap as measured, every loaded entry's estimated footprint, and their sum.
The memory picture the browser shows: the JVM heap as measured, every loaded entry's estimated footprint, and their sum.
(name-of kb)What to call kb — the name of the entry holding it (entry-of), or nil.
A page that has resolved a KB and judged it asks this rather than active-entry:
activate re-points the holder at any moment and takes no monitor, so the active entry
a moment later can be a different KB — and a refusal that named it would name the KB it
did not judge.
What to call `kb` — the name of the entry holding it (`entry-of`), or nil. A page that has *resolved* a KB and judged it asks this rather than `active-entry`: `activate` re-points the holder at any moment and takes no monitor, so the active entry a moment later can be a different KB — and a refusal that named it would name the KB it did not judge.
(predicted-footprint {:keys [total]})What a source would cost in RAM if it were loaded, in bytes — its own count of what it holds put through the same coefficients — or nil for a source that does not know how big it is (the generator, whose size is whatever the sliders say).
A load builds belief and holds its records in RAM unless a directory is named, so this is the full three-component figure: the ceiling, not the floor.
What a source would cost in RAM if it were loaded, in bytes — its own count of what it holds put through the same coefficients — or nil for a source that does not know how big it is (the generator, whose size is whatever the sliders say). A load builds belief and holds its records in RAM unless a directory is named, so this is the full three-component figure: the ceiling, not the floor.
(register! key name kb)(register! key name kb {:keys [where source]})File an already-built KB as a settled entry and make it active if nothing else is — how the browser's own startup KB, and an attached daemon, get into the list beside the ones the catalog loads.
opts: :where says what unloading it should release (nil for a KB this process does
not own — an attached daemon is nobody's to close); :source names the source it came
from, so a KB registered at startup shows as loaded rather than being offered again.
File an already-built KB as a settled entry and make it active if nothing else is — how the browser's own startup KB, and an attached daemon, get into the list beside the ones the catalog loads. `opts`: `:where` says what unloading it should release (nil for a KB this process does not own — an attached daemon is nobody's to close); `:source` names the source it came from, so a KB registered at startup shows as *loaded* rather than being offered again.
(released? kb)Did unload! release kb's stores (clear a memory KB, or close a disk one)? A write
parked on the browser's write monitor while an unload released its KB asks this once it
holds the monitor, and is refused. The question is "was this KB released", not "does
the catalog hold it": a KB the catalog never held is never refused. An unload that
releases nothing marks nothing, and register! clears the mark of a KB it files again.
Did `unload!` release `kb`'s stores (clear a memory KB, or close a disk one)? A write parked on the browser's write monitor while an unload released its KB asks this once it holds the monitor, and is refused. The question is "was this KB released", not "does the catalog hold it": a KB the catalog never held is never refused. An unload that releases nothing marks nothing, and `register!` clears the mark of a KB it files again.
(request-paths?)May a request name a directory for this process to write (VAELII_REQUEST_PATHS,
default off)? Off, a durable load's store and an export's dump go under write-root
with a name server-dir picks, and the browser refuses a request that names one.
May a request name a directory for this process to write (`VAELII_REQUEST_PATHS`, default off)? Off, a durable load's store and an export's dump go under `write-root` with a name `server-dir` picks, and the browser refuses a request that names one.
(reset-registry!)Forget every entry, releasing each as unload! does, and stop every job. For tests:
nothing in the browser calls it, and no shutdown hook does. Every running job is stopped
first and waited for, since unload! refuses a KB under a live export or writer
(:still-exporting, :still-writing). The cancels are issued together and the waits
overlap, each bounded by 30 s; a job that does not stop keeps its KB. One entry refusing
to release does not stop the rest: each refusal is logged and the sweep goes on.
Forget every entry, releasing each as `unload!` does, and stop every job. For tests: nothing in the browser calls it, and no shutdown hook does. Every running job is stopped **first** and waited for, since `unload!` refuses a KB under a live export or writer (`:still-exporting`, `:still-writing`). The cancels are issued together and the waits overlap, each bounded by 30 s; a job that does not stop keeps its KB. One entry refusing to release does not stop the rest: each refusal is logged and the sweep goes on.
What one stored sentex costs in RAM, per resident component: the coefficients
footprint multiplies out, measured on a real-assert load through bench-scale and
bench-memory. Resident size is linear in the sentex count, so the estimate takes one
coefficient per component. The main error term is the sentence's shape: a
fact of arity 2 with no compound arguments indexes at ~1,549 B and a richer one at
~2,158 B; the leaner figure is used, so a corpus of fat sentences reads low. The :tms
figure is the dense network's, the default (the reference map is ~467 B/sentex,
~3.8× more): 18 B/node + 166 B/justification, so ~101 per stored sentex at j/n≈0.5. A
justification-heavy KB reads low; the coefficient never counts justifications on the
render path.
What one stored sentex costs in RAM, per resident component: the coefficients `footprint` multiplies out, measured on a real-assert load through `bench-scale` and `bench-memory`. Resident size is linear in the sentex count, so the estimate takes one coefficient per component. The main error term is the sentence's shape: a fact of arity 2 with no compound arguments indexes at ~1,549 B and a richer one at ~2,158 B; the leaner figure is used, so a corpus of fat sentences reads low. The `:tms` figure is the **dense** network's, the default (the reference map is ~467 B/sentex, ~3.8× more): `18 B/node + 166 B/justification`, so ~101 per stored sentex at j/n≈0.5. A justification-heavy KB reads low; the coefficient never counts justifications on the render path.
(search-path)The directories discovery walks: VAELII_KB_PATH when set, else the
vaelii.kb.path system property, else ./kbs and ~/.vaelii/kbs. The list is
separated as the platform's PATH is — :, and ; on Windows, where : is part of
every absolute path. A path entry that
is a KB directory counts as one source; otherwise its children are probed, one level
down. (The property mirrors vaelii.disk.dir, and is what a test sets — a JVM cannot
change its own environment.) Either spelling blank is unset — set-to for why.
The directories discovery walks: `VAELII_KB_PATH` when set, else the `vaelii.kb.path` system property, else `./kbs` and `~/.vaelii/kbs`. The list is separated as the platform's `PATH` is — `:`, and `;` on Windows, where `:` is part of every absolute path. A path entry that *is* a KB directory counts as one source; otherwise its children are probed, one level down. (The property mirrors `vaelii.disk.dir`, and is what a test sets — a JVM cannot change its own environment.) Either spelling **blank** is unset — `set-to` for why.
(source id)The source with this id, or nil.
The source with this id, or nil.
(sources)Every KB this process can load, built-ins first: the shipped ontologies and the
generator, then the catalog file's entries, then the search path's. Recomputed per call,
so a corpus dropped into a search-path directory appears on the next page load. A
search-path entry holding more than max-discovered candidates is probed for the first
max-discovered by name, and what was passed over is named on the vector's metadata
as :truncated and in the log. The built-ins and the catalog file's entries are never
capped.
Every KB this process can load, built-ins first: the shipped ontologies and the generator, then the catalog file's entries, then the search path's. Recomputed per call, so a corpus dropped into a search-path directory appears on the next page load. A search-path entry holding more than `max-discovered` candidates is probed for the first `max-discovered` by name, and what was passed over is **named** on the vector's metadata as `:truncated` and in the log. The built-ins and the catalog file's entries are never capped.
(stats kb)The headline counts for a loaded KB. term-count is one set-size read and the
context sizes are one each, so this is cheap even on a corpus of millions — deliberately
so, since it runs every time the page lists the entries.
Nil for anything that is not an in-process KB (an attached daemon is registered as an entry too, and its counts are the daemon's to report).
The headline counts for a loaded KB. `term-count` is one set-size read and the context sizes are one each, so this is cheap even on a corpus of millions — deliberately so, since it runs every time the page lists the entries. Nil for anything that is not an in-process KB (an attached daemon is registered as an entry too, and its counts are the daemon's to report).
(sync-entry! key)Sync the shipped contexts of entry key as a :sync job, whatever its load asked,
one change at a time with each reported, filing the report on it as :sync
(sync-now!), and return the job id. How a store
or dump loaded without :sync? is brought current on purpose. Throws
:unknown-source when no entry key holds a KB in this process, :unrecovered-kb when
that KB accepts no writes (v/write-hazards), and {:type :job-busy} while another job
holds this process's writer.
Sync the shipped contexts of entry `key` as a `:sync` job, whatever its load asked,
one change at a time with each reported, filing the report on it as `:sync`
(`sync-now!`), and return the job id. How a store
or dump loaded without `:sync?` is brought current on purpose. Throws
`:unknown-source` when no entry `key` holds a KB in this process, `:unrecovered-kb` when
that KB accepts no writes (`v/write-hazards`), and `{:type :job-busy}` while another job
holds this process's writer.(sync-loaded!)Sync the shipped contexts of every entry that holds a KB in this process and accepts
writes, as one :sync job: the shipped files are read once (sync/premises), and
each entry's report is filed on it as :sync (sync-now!). What the development
browser runs when a shipped KB file changes on disk (vaelii.browser.web). A store or
dump loaded without :sync? is not one of them (follows-shipped?). Returns the job
id, or nil when no entry qualifies. Throws {:type :job-busy} while another job holds
this process's writer.
Sync the shipped contexts of every entry that holds a KB in this process and accepts
writes, as one `:sync` job: the shipped files are read once (`sync/premises`), and
each entry's report is filed on it as `:sync` (`sync-now!`). What the development
browser runs when a shipped KB file changes on disk (`vaelii.browser.web`). A store or
dump loaded without `:sync?` is not one of them (`follows-shipped?`). Returns the job
id, or nil when no entry qualifies. Throws `{:type :job-busy}` while another job holds
this process's writer.(unload! key)(unload! key {:keys [run-in]})Take an entry down: cancel it if it is still loading, release what it held, and drop it
from the registry. A memory-backed KB is cleared; every other backend is closed,
not cleared (v/close!), so the file lock is released and the directory is left as it
was, and an adapter store's connection is closed too. An entry register! filed with no
:where releases nothing. The ! is for the memory case.
Four ways it declines, each about a KB something else still holds:
:still-stopping, retryable, since cancellation is
cooperative.:still-writing, naming the job. A chaining run or a
shipped-context sync claims the KB as its writer. Asked before :run-in is
entered, so the answer does not wait for a fixpoint, and again inside for a job
registered meanwhile. Refused, not cancelled, since a daemon's chain reports no
progress.:still-exporting. That test and the release are one
step under start-monitor, the monitor export-entry! checks and submits under;
otherwise an unload and an export arriving together each pass their own check and the
walk dumps an emptied KB. The entry is dropped inside the monitor too.:unreleased and
the reason on it, is not active, and the throw reaches the caller. Unloading again
retries the release.opts takes :run-in, a wrapper the check-and-release step runs inside (as
export-entry!'s): the browser hands its write monitor, so a write already inside
finishes before the stores go, and a write still waiting enters after the release and is
refused by released?. :run-in is entered before start-monitor is taken. The
write monitor is process-wide, so an unload still waits for a chain running on another
KB.
Take an entry down: cancel it if it is still loading, release what it held, and drop it from the registry. **A memory-backed KB is cleared**; **every other backend is closed, not cleared** (`v/close!`), so the file lock is released and the directory is left as it was, and an adapter store's connection is closed too. An entry `register!` filed with no `:where` releases nothing. The `!` is for the memory case. Four ways it declines, each about a KB something else still holds: - **its loader has not stopped**: `:still-stopping`, retryable, since cancellation is cooperative. - **another job writes it**: `:still-writing`, naming the job. A chaining run or a shipped-context sync claims the KB as its writer. Asked **before** `:run-in` is entered, so the answer does not wait for a fixpoint, and again inside for a job registered meanwhile. Refused, not cancelled, since a daemon's chain reports no progress. - **an export is walking it**: `:still-exporting`. **That test and the release are one step under `start-monitor`**, the monitor `export-entry!` checks and submits under; otherwise an unload and an export arriving together each pass their own check and the walk dumps an emptied KB. The entry is dropped inside the monitor too. - **the release itself failed**: the entry keeps its place with status `:unreleased` and the reason on it, is not active, and the throw reaches the caller. Unloading again retries the release. `opts` takes `:run-in`, a wrapper the check-and-release step runs inside (as `export-entry!`'s): the browser hands its write monitor, so a write already inside finishes before the stores go, and a write still waiting enters after the release and is refused by `released?`. `:run-in` is entered **before** `start-monitor` is taken. The write monitor is process-wide, so an unload still waits for a chain running on another KB.
(write-back! key base)Write to the shipped files what entry key's KB holds in its exact contexts and base
does not, and remove what base holds and the KB does not (sync/write-back!).
base is an earlier held-premises of the same entry. Returns [held report], the
report nil when nothing changed, or nil when held-premises is.
Write to the shipped files what entry `key`'s KB holds in its exact contexts and `base` does not, and remove what `base` holds and the KB does not (`sync/write-back!`). `base` is an earlier `held-premises` of the same entry. Returns `[held report]`, the report nil when nothing changed, or nil when `held-premises` is.
(write-blocked? kb)Is kb one a loader is still writing? Reading beside a loader is sound; writing
beside one is not, since two interleaved writers are not serializable and this process's
one writer is the loader (docs/storage.md). Asked of the KB itself, by identity, not
of the active entry: loading a second KB is no reason to stop writing to the one on
screen, and a KB the catalog never heard of is nobody's loader's business. Two halves: a
job handed a KB (a chaining run) says so at submit (jobs/writes-kb?), and a load
opens its own, so the identity is on the entry.
Is `kb` one a loader is still **writing**? Reading beside a loader is sound; writing beside one is not, since two interleaved writers are not serializable and this process's one writer is the loader (docs/storage.md). Asked of the **KB itself**, by identity, not of the active entry: loading a second KB is no reason to stop writing to the one on screen, and a KB the catalog never heard of is nobody's loader's business. Two halves: a job handed a KB (a chaining run) says so at `submit` (`jobs/writes-kb?`), and a **load** opens its own, so the identity is on the entry.
(write-root)The directory a store or a dump the request did not name goes under: the first
search-path entry that is not itself a KB, so a dump written there is a :dump source
the moment it lands. Nil when every entry is a KB.
The directory a store or a dump the request did not name goes under: the first search-path entry that is not itself a KB, so a dump written there is a `:dump` source the moment it lands. Nil when every entry is a KB.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |