Long work, as jobs: one registry, one progress reading, one cancel.
Three things this process does take minutes rather than milliseconds — filling a KB from a corpus, writing one back out, and joining every rule over everything stored. Each of them wants the same four capabilities, and they are the only four: run on a thread of its own so the pages keep answering, say where it has got to, stop when asked, and leave a report somebody can read afterwards. That shape is here once.
A job is {:id :label :kind :status :progress :started :finished :error :summary :result-url}, plus a cancel flag and the future, which no view carries. submit
returns the id; job reads one; jobs lists them, newest first. The caller's work
is handed a progress! fn and nothing else: what it records shows up under
:progress, and what it throws is how cancellation lands, because a tight assert
loop has no other point at which stopping is safe.
One status vocabulary, whatever the job is doing:
:running → :cancelling → :done | :cancelled | :failed
:cancelling is the honest middle: cancel! sets the flag and returns, and the work
keeps running until it reaches its next progress report — which, for a phase that
reports none (opening a large store scans its whole record log before it says
anything), can be a while.
The single writer stays single. :writes names the KB a job writes, or true for
one it has not opened yet, and one writing job runs at a time: a second is refused
with a message naming the job that holds the writer. Two interleaved writers are not
serializable (docs/storage.md, the single-writer contract), and a registry that let two
through would be a way around the contract rather than a place to watch it from.
writes-kb? is the other half of the same question, asked by identity, so a job filling
one KB never blocks a write to another.
Cancellation is cooperative, and for a KB-writing job that is not negotiable. A
thread interrupt landing mid-cascade on a durable store surfaces as
ClosedByInterruptException and can leave a torn removal, so a job with :writes is
flagged and never interrupted however long it takes to notice. A job that writes
nothing may say :interruptible? true and be cancelled the hard way as well; the
registry checks both, so the two can never be confused for one another.
A finished job's report outlives the job, for an hour — long enough to read what it did, since the page that would have shown it is usually the page you navigated away from. Nothing unsettled is ever dropped, at any age: forgetting a job is releasing its writer claim, and a thread that is still running is still writing. So a wedged job keeps its place and keeps counting towards the running badge, which is the truth about the process — better than a store two writers took turns on.
Long work, as jobs: one registry, one progress reading, one cancel.
Three things this process does take minutes rather than milliseconds — filling a KB
from a corpus, writing one back out, and joining every rule over everything stored.
Each of them wants the same four capabilities, and they are the only four: run on a
thread of its own so the pages keep answering, say where it has got to, stop when
asked, and leave a report somebody can read afterwards. That shape is here once.
**A job** is `{:id :label :kind :status :progress :started :finished :error :summary
:result-url}`, plus a cancel flag and the future, which no view carries. `submit`
returns the id; `job` reads one; `jobs` lists them, newest first. The caller's `work`
is handed a `progress!` fn and nothing else: what it records shows up under
`:progress`, and what it *throws* is how cancellation lands, because a tight assert
loop has no other point at which stopping is safe.
**One status vocabulary**, whatever the job is doing:
:running → :cancelling → :done | :cancelled | :failed
`:cancelling` is the honest middle: `cancel!` sets the flag and returns, and the work
keeps running until it reaches its next progress report — which, for a phase that
reports none (opening a large store scans its whole record log before it says
anything), can be a while.
**The single writer stays single.** `:writes` names the KB a job writes, or `true` for
one it has not opened yet, and **one writing job runs at a time**: a second is refused
with a message naming the job that holds the writer. Two interleaved writers are not
serializable (docs/storage.md, the single-writer contract), and a registry that let two
through would be a way around the contract rather than a place to watch it from.
`writes-kb?` is the other half of the same question, asked by identity, so a job filling
one KB never blocks a write to another.
**Cancellation is cooperative, and for a KB-writing job that is not negotiable.** A
thread interrupt landing mid-cascade on a durable store surfaces as
`ClosedByInterruptException` and can leave a torn removal, so a job with `:writes` is
flagged and never interrupted however long it takes to notice. A job that writes
nothing may say `:interruptible? true` and be cancelled the hard way as well; the
registry checks both, so the two can never be confused for one another.
**A finished job's report outlives the job**, for an hour — long enough to read what it
did, since the page that would have shown it is usually the page you navigated away
from. Nothing *unsettled* is ever dropped, at any age: forgetting a job is releasing
its writer claim, and a thread that is still running is still writing. So a wedged job
keeps its place and keeps counting towards the running badge, which is the truth about
the process — better than a store two writers took turns on.(cancel! id)Ask job id to stop at its next progress report, and answer whether there was one to
ask. ! because of what a stopped job leaves behind: a KB holding a prefix of a load
or of a chaining run, or a directory holding part of a dump.
A job that writes a KB is flagged and never interrupted, however long it takes to
notice — an interrupt landing mid-cascade on a durable store tears the write it lands
in. A job that writes nothing and says :interruptible? true is flagged and
interrupted, so it unwinds promptly rather than at a report it may not reach. Both are
checked, not one: a job that says it may be interrupted and writes a KB anyway is not
interrupted, since the reason is about the store rather than about the promise.
Ask job `id` to stop at its next progress report, and answer whether there was one to ask. `!` because of what a stopped job leaves behind: a KB holding a prefix of a load or of a chaining run, or a directory holding part of a dump. A job that writes a KB is flagged and **never** interrupted, however long it takes to notice — an interrupt landing mid-cascade on a durable store tears the write it lands in. A job that writes nothing and says `:interruptible? true` is flagged *and* interrupted, so it unwinds promptly rather than at a report it may not reach. Both are checked, not one: a job that says it may be interrupted and writes a KB anyway is not interrupted, since the reason is about the store rather than about the promise.
(cancelled? t)Was t thrown by a cancellation rather than by a failure — the progress! throw
cancel! arms, or the interrupt it sends a job that writes nothing?
Public because a caller filing a status of its own beside the registry's — the catalog,
onto the entry a load leaves behind — has to classify a throw exactly as submit does,
and the alternative is this literal written in two places.
Was `t` thrown by a **cancellation** rather than by a failure — the `progress!` throw `cancel!` arms, or the interrupt it sends a job that writes nothing? Public because a caller filing a status of its own beside the registry's — the catalog, onto the entry a load leaves behind — has to classify a throw exactly as `submit` does, and the alternative is this literal written in two places.
How long a caller may wait for a job before answering with a progress page instead of its result. A quarter of a second: the point is that a small operation does not acquire a spinner and a second round trip, since a tool where every action costs one feels slower than the thing it replaced.
How long a caller may wait for a job before answering with a progress page instead of its result. A quarter of a second: the point is that a small operation does **not** acquire a spinner and a second round trip, since a tool where every action costs one feels slower than the thing it replaced.
(job id)Job id, as a view, or nil once it has been swept.
Job `id`, as a view, or nil once it has been swept.
(jobs)Every job the registry still holds, newest first — the order /jobs lists them in.
Every job the registry still holds, newest first — the order `/jobs` lists them in.
(latest kind)The newest job of kind, settled or not — the last export's report is what the export
panel shows, and it is worth keeping visible until the next one replaces it.
The newest job of `kind`, settled or not — the last export's report is what the export panel shows, and it is worth keeping visible until the next one replaces it.
(reset-registry!)Cancel every job and forget them all. For a process shutting down and for tests; nothing in the browser calls it.
Cancel every job and forget them all. For a process shutting down and for tests; nothing in the browser calls it.
(running)The jobs that have not settled, newest first. What a panel polls on, and what the running badge counts.
The jobs that have not settled, newest first. What a panel polls on, and what the running badge counts.
(submit {:keys [label kind writes progress result-url] :as opts} work)Run work as a job and return its id. work takes one argument, the progress! fn
above, and its return value is filed as the job's :summary (a map may carry a
:result-url, which then overrides the one submitted).
opts:
| key | |
|---|---|
:label | what the job is called on screen |
:kind | :load / :export / :chain — what a panel filters on |
:writes | the KB this job writes, or true for one it will open |
:interruptible? | may its thread be interrupted on cancel? Only for a job that writes nothing |
:progress | the first progress reading, before the work has said anything |
:result-url | where to send a reader when it finishes |
Any other key is carried through onto the job, which is how the export's destination and the entry's name reach the panel that renders them.
Throws {:type :job-busy} when :writes is asked for and another job already holds
the writer. The refusal names the holder rather than queueing behind it: two writers
are not serializable, and a queue would make a load's wall-clock reading mean whatever
was in front of it.
Run `work` as a job and return its id. `work` takes one argument, the `progress!` fn
above, and its return value is filed as the job's `:summary` (a map may carry a
`:result-url`, which then overrides the one submitted).
`opts`:
| key | |
|---|---|
| `:label` | what the job is called on screen |
| `:kind` | `:load` / `:export` / `:chain` — what a panel filters on |
| `:writes` | the KB this job writes, or `true` for one it will open |
| `:interruptible?` | may its thread be interrupted on cancel? Only for a job that writes nothing |
| `:progress` | the first progress reading, before the work has said anything |
| `:result-url` | where to send a reader when it finishes |
Any other key is carried through onto the job, which is how the export's destination
and the entry's name reach the panel that renders them.
Throws `{:type :job-busy}` when `:writes` is asked for and another job already holds
the writer. The refusal names the holder rather than queueing behind it: two writers
are not serializable, and a queue would make a load's wall-clock reading mean whatever
was in front of it.(wait id ms)Block up to ms for job id to settle, then answer its view — settled or not, so the
caller decides what to do about a job that is still going. This is what the fast path
is: submit, wait fast-path-ms, and answer with the result if it is already there.
Block up to `ms` for job `id` to settle, then answer its view — settled or not, so the caller decides what to do about a job that is still going. This is what the fast path is: submit, wait `fast-path-ms`, and answer with the result if it is already there.
(writer)The running job that holds this process's writer, or nil. What submit refuses a
second writing job against, and what a refusal names.
The running job that holds this process's writer, or nil. What `submit` refuses a second writing job against, and what a refusal names.
(writes-kb? kb)Is a job writing this KB, by identity? Reading beside a writer is sound and writing beside one is not, and the question is about the KB rather than about the process: a job filling one KB is no reason to refuse a write to another.
Is a job writing **this** KB, by identity? Reading beside a writer is sound and writing beside one is not, and the question is about the KB rather than about the process: a job filling one KB is no reason to refuse a write to another.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |