Seals and restores for a KB over :disk or :pg records with the :snapshot index
(:disk-snapshot, :pg-snapshot) that records its writes in an operation log
(vaelii.impl.oplog).
seal! writes the KB's derived state and starts the log again: the index image
(vaelii.impl.disk.index-snapshot), the reasoning image (vaelii.impl.reasoning-image), then
<dir>/oplog/seal.nippy, then a new generation of the log. It fsyncs a disk record
store and the reasoning image's three files before it writes seal.nippy, so every
record below the watermark, and both images, are on disk once a seal names them; a
Postgres record is durable when its statement commits. seal.nippy holds the
generation, the records watermark — one above every handle the store held when the seal
was taken — and the two record fingerprints the images are written with
(p/ImageFingerprinting). It is written after both images and before the log
restarts, so each crash point leaves a state an open can tell apart:
seal.nippy: the images carry fingerprints the previous seal does not, so
neither installs against it, and the open rebuilds from the records;seal.nippy and before the restart: the log's header names the previous
generation, so its frames describe writes both images already hold, and the open
replays none of them.A KB is sealed when a :seal-class operation returns, when its directory closes, when
its index drifts past vaelii.index.snapshot-drift, and when vaelii.core/seal is
called: the drift is measured inside a write, so the seal waits until the operation
returns (oplog/request-seal!). A seal is declined, and the log left as it is, inside
an operation and while a settle is running or deferred (busy). A KB
reasoning-image/refusal names a reason for, or whose network does not cover its records,
cannot be sealed, and its log is marked unusable instead.
restore! takes a KB opened with {:recover? false} over the directory and brings it
to the state its last durable operation left. It installs both images against the
fingerprints seal.nippy records, attaches the log in replay mode, and replays the
current generation's frames (oplog/replay!). Under kb/*trust-source?* a reasoning
image written under other engine source installs and is kept, as :recover? :trust
keeps one (recovery/trust-installed!), and the seals after it record that source. It
declines — returning a reason and leaving the caller to rebuild from the records — when
there is no seal, the log is unusable, an image does not install, a replayed write
differs from the record stored at its handle, or the store holds a record at or above
the last handle the replay allocated, which a write whose frame never reached the disk
left behind.
A decline after the images installed leaves replayed state in the KB. restore! then
withdraws the directory's image writers and notes the :no-belief and :no-index
hazards (kb/note-hazards!), so closing that KB writes no image of the state.
open! is vaelii.core/open-kb under :oplog?: a directory with a seal is opened
{:recover? false} and restored, trusting the image under :recover? :trust, and on a
decline it is closed and opened again with the caller's :recover?; a directory with
none is opened and attached. The report of which path ran is logged and kept on the log
(oplog/opened).
Seals and restores for a KB over `:disk` or `:pg` records with the `:snapshot` index
(`:disk-snapshot`, `:pg-snapshot`) that records its writes in an operation log
(`vaelii.impl.oplog`).
## A seal
`seal!` writes the KB's derived state and starts the log again: the index image
(`vaelii.impl.disk.index-snapshot`), the reasoning image (`vaelii.impl.reasoning-image`), then
`<dir>/oplog/seal.nippy`, then a new generation of the log. It fsyncs a disk record
store and the reasoning image's three files before it writes `seal.nippy`, so every
record below the watermark, and both images, are on disk once a seal names them; a
Postgres record is durable when its statement commits. `seal.nippy` holds the
generation, the records watermark — one above every handle the store held when the seal
was taken — and the two record fingerprints the images are written with
(`p/ImageFingerprinting`). It is written after both images and before the log
restarts, so each crash point leaves a state an open can tell apart:
- before `seal.nippy`: the images carry fingerprints the previous seal does not, so
neither installs against it, and the open rebuilds from the records;
- after `seal.nippy` and before the restart: the log's header names the previous
generation, so its frames describe writes both images already hold, and the open
replays none of them.
A KB is sealed when a `:seal`-class operation returns, when its directory closes, when
its index drifts past `vaelii.index.snapshot-drift`, and when `vaelii.core/seal` is
called: the drift is measured inside a write, so the seal waits until the operation
returns (`oplog/request-seal!`). A seal is declined, and the log left as it is, inside
an operation and while a settle is running or deferred (`busy`). A KB
`reasoning-image/refusal` names a reason for, or whose network does not cover its records,
cannot be sealed, and its log is marked unusable instead.
## A restore
`restore!` takes a KB opened with `{:recover? false}` over the directory and brings it
to the state its last durable operation left. It installs both images against the
fingerprints `seal.nippy` records, attaches the log in replay mode, and replays the
current generation's frames (`oplog/replay!`). Under `kb/*trust-source?*` a reasoning
image written under other engine source installs and is kept, as `:recover? :trust`
keeps one (`recovery/trust-installed!`), and the seals after it record that source. It
declines — returning a reason and leaving the caller to rebuild from the records — when
there is no seal, the log is unusable, an image does not install, a replayed write
differs from the record stored at its handle, or the store holds a record at or above
the last handle the replay allocated, which a write whose frame never reached the disk
left behind.
A decline after the images installed leaves replayed state in the KB. `restore!` then
withdraws the directory's image writers and notes the `:no-belief` and `:no-index`
hazards (`kb/note-hazards!`), so closing that KB writes no image of the state.
## An open
`open!` is `vaelii.core/open-kb` under `:oplog?`: a directory with a seal is opened
`{:recover? false}` and restored, trusting the image under `:recover? :trust`, and on a
decline it is closed and opened again with the caller's `:recover?`; a directory with
none is opened and attached. The report of which path ran is logged and kept on the log
(`oplog/opened`).(attach! kb)(attach! kb opts)kb — a :disk-snapshot or :pg-snapshot KB whose belief and index cover its
records — recording its writes into a log under its directory, sealed once so the log
starts at a generation its images describe. Returns the logged KB. opts is
oplog/open-log's.
`kb` — a `:disk-snapshot` or `:pg-snapshot` KB whose belief and index cover its records — recording its writes into a log under its directory, sealed once so the log starts at a generation its images describe. Returns the logged KB. `opts` is `oplog/open-log`'s.
(busy kb)Why kb cannot be sealed now and can be later, or nil: :in-operation inside a
recorded operation on this thread, :settle-deferred inside a with-deferred-settle
batch whose settle has not run, :settle-running while a settle holds the network.
Why `kb` cannot be sealed now and can be later, or nil: `:in-operation` inside a recorded operation on this thread, `:settle-deferred` inside a `with-deferred-settle` batch whose settle has not run, `:settle-running` while a settle holds the network.
seal.nippy's layout number. A seal of any other number is not restored. Version 3
names the index image's fingerprint :index-fp and the reasoning image's
:reasoning-fp, each the record store's p/ImageFingerprinting answer.
`seal.nippy`'s layout number. A seal of any other number is not restored. Version 3 names the index image's fingerprint `:index-fp` and the reasoning image's `:reasoning-fp`, each the record store's `p/ImageFingerprinting` answer.
(open! opts open-fn log-opts)Open the :disk-snapshot or :pg-snapshot KB opts describes with an operation log
attached (the namespace docstring, "An open"). open-fn opens a KB from an opts map,
as vaelii.core/open-kb does without :oplog?; log-opts is oplog/open-log's.
Returns the logged KB; oplog/opened of its log is {:restored true :frames n :ms t} or
{:restored false :reason r :ms t}.
Open the `:disk-snapshot` or `:pg-snapshot` KB `opts` describes with an operation log
attached (the namespace docstring, "An open"). `open-fn` opens a KB from an opts map,
as `vaelii.core/open-kb` does without `:oplog?`; `log-opts` is `oplog/open-log`'s.
Returns the logged KB; `oplog/opened` of its log is `{:restored true :frames n :ms t}` or
`{:restored false :reason r :ms t}`.(read-seal dir)The seal dir holds, or nil.
The seal `dir` holds, or nil.
(restore! kb)(restore! kb opts)Restore kb, opened {:recover? false} over its directory (the namespace docstring,
"A restore"). Returns {:restored true :kb lkb :frames n}, or {:restored false :reason r :clean? b}: :clean? false when the KB already holds replayed state and
the caller rebuilds by reopening rather than by recovering this value. A throw
declines as an unclean decline does — the log closed and the image writers withdrawn —
before it propagates. opts is oplog/open-log's.
Restore `kb`, opened `{:recover? false}` over its directory (the namespace docstring,
"A restore"). Returns `{:restored true :kb lkb :frames n}`, or `{:restored false
:reason r :clean? b}`: `:clean?` false when the KB already holds replayed state and
the caller rebuilds by reopening rather than by recovering this value. A throw
declines as an unclean decline does — the log closed and the image writers withdrawn —
before it propagates. `opts` is `oplog/open-log`'s.(retire-unlogged! kb)Rename the seal of kb's directory to retired.nippy before kb, which keeps no
operation log, writes. The write reaches the records and no log, so a restore from
that seal would answer without it; with no seal, the next logged open opens from the
images or the records and seals again, at a generation above the retired one. Any
durable directory counts, not only an image's: a non-snapshot disk backend opened over
a sealed directory writes the records and leaves the seal's images as they were. One
file stat on a KB with a directory, and nothing on any other.
Rename the seal of `kb`'s directory to `retired.nippy` before `kb`, which keeps no operation log, writes. The write reaches the records and no log, so a restore from that seal would answer without it; with no seal, the next logged open opens from the images or the records and seals again, at a generation above the retired one. Any durable directory counts, not only an image's: a non-snapshot disk backend opened over a sealed directory writes the records and leaves the seal's images as they were. One file stat on a KB with a directory, and nothing on any other.
(seal! kb)Seal kb (the namespace docstring, "A seal"). Returns {:sealed true :generation g :watermark w :ms t}, or {:sealed false :reason r}, with :busy true when r is one
of busy's and the log is left as it is. A log whose fault is latched refuses with
:store-unusable before anything is written.
Seal `kb` (the namespace docstring, "A seal"). Returns `{:sealed true :generation g
:watermark w :ms t}`, or `{:sealed false :reason r}`, with `:busy true` when `r` is one
of `busy`'s and the log is left as it is. A log whose fault is latched refuses with
`:store-unusable` before anything is written.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |