The output of ONE background shell, stored as a FILE and read by BYTE OFFSET.
The file is the STORAGE and the in-memory ring buffer is only a VIEW. A ring
is a display convenience: it answers "what is on screen now", and the moment a
command prints more than the ring holds, the head is gone before the first
poll and no sequence of reads can recover it. That is the whole reported bug,
and it is a storage bug, so the fix is storage: every byte the pump reads is
appended to ~/.vis/logs/shell/<session>/<id>.log, and a read names the byte
it starts at.
A chunk is the cat contract for a growing file, key for key: give an
offset, get the bytes and the next-offset to continue from. Feeding
next-offset back in a loop yields the WHOLE stream with no overlap and no
gap, which is why there is no dropped count anywhere in this namespace —
nothing is dropped, so nothing has to be reported as lost.
is-eof means "you have read everything WRITTEN so far", never "the command
finished": the process's own status belongs to its handle, not to a read of
its log. is-truncated is a cap on THIS read alone and never on the file.
The log is PERSISTENT and belongs to the SESSION: it outlives the process's
exit, a daemon restart, and the turn that started the command. It dies with
the session — delete-session-logs! runs where the session record is
deleted, and the DB index row is scoped to the session soul, so the database
cascade retires it in the same breath.
Bytes on disk, index in the DB. The log never becomes a row: it is an
append-only stream read by offset, and sqlite would turn every pump flush into
a blob rewrite and every cursor read into a substring over that blob. What
the DB carries is the ROW that makes a log FINDABLE without holding a
handle — the command, the path, the start/end and the exit — on the
extension_aggregate sidecar rail under index-extension-id.
The output of ONE background shell, stored as a FILE and read by BYTE OFFSET. The file is the STORAGE and the in-memory ring buffer is only a VIEW. A ring is a display convenience: it answers "what is on screen now", and the moment a command prints more than the ring holds, the head is gone before the first poll and no sequence of reads can recover it. That is the whole reported bug, and it is a storage bug, so the fix is storage: every byte the pump reads is appended to `~/.vis/logs/shell/<session>/<id>.log`, and a read names the byte it starts at. A chunk is the `cat` contract for a growing file, key for key: give an `offset`, get the bytes and the `next-offset` to continue from. Feeding `next-offset` back in a loop yields the WHOLE stream with no overlap and no gap, which is why there is no `dropped` count anywhere in this namespace — nothing is dropped, so nothing has to be reported as lost. `is-eof` means "you have read everything WRITTEN so far", never "the command finished": the process's own status belongs to its handle, not to a read of its log. `is-truncated` is a cap on THIS read alone and never on the file. The log is PERSISTENT and belongs to the SESSION: it outlives the process's exit, a daemon restart, and the turn that started the command. It dies with the session — [[delete-session-logs!]] runs where the session record is deleted, and the DB index row is scoped to the session soul, so the database cascade retires it in the same breath. Bytes on disk, index in the DB. The log never becomes a row: it is an append-only stream read by offset, and sqlite would turn every pump flush into a blob rewrite and every cursor read into a substring over that blob. What the DB carries is the ROW that makes a log FINDABLE without holding a handle — the command, the path, the start/end and the exit — on the `extension_aggregate` sidecar rail under [[index-extension-id]].
(close! sink)Flush and close a sink. Idempotent and never throws — a log that cannot be closed must not take the shell's teardown down with it.
Flush and close a sink. Idempotent and never throws — a log that cannot be closed must not take the shell's teardown down with it.
Bytes ONE read returns when the caller named no limit. A window, not the
file: the rest is one more read away at next-offset.
Bytes ONE read returns when the caller named no limit. A window, not the file: the rest is one more read away at `next-offset`.
(delete-session-logs! session)Delete every shell log of session. The retention rule: a log dies with the
session that produced it, and with nothing else — a build log is large and
boring the day after, but only the session knows when that day came.
Delete every shell log of `session`. The retention rule: a log dies with the session that produced it, and with nothing else — a build log is large and boring the day after, but only the session knows when that day came.
(index! db-info session id data)Upsert the row that makes ONE log findable by session: the command, the log path, the start, and — once the pump has seen the child die — the end and the exit code. Best effort by construction: the bytes are on disk either way, and an index that throws must never take a running shell with it.
data is engine-shaped (kebab keywords) and reaches the database through
[[wire/->wire]], because a row that outlives the process is JSON on disk and
wears the wire's snake_case string keys — which is exactly what
session-logs reads back.
Upsert the row that makes ONE log findable by session: the command, the log path, the start, and — once the pump has seen the child die — the end and the exit code. Best effort by construction: the bytes are on disk either way, and an index that throws must never take a running shell with it. `data` is engine-shaped (kebab keywords) and reaches the database through [[wire/->wire]], because a row that outlives the process is JSON on disk and wears the wire's snake_case string keys — which is exactly what [[session-logs]] reads back.
Owner of the sidecar rows. The shell extension owns its own index.
Owner of the sidecar rows. The shell extension owns its own index.
kind of a shell-log sidecar row.
`kind` of a shell-log sidecar row.
(log-file session id)The log file of shell id in session. Deterministic, so the bytes are
reachable with cat and grep like any other file.
The log file of shell `id` in `session`. Deterministic, so the bytes are reachable with `cat` and `grep` like any other file.
Ceiling on ONE read, so a caller cannot bill a 40 MiB build log to a single context window by asking for it.
Ceiling on ONE read, so a caller cannot bill a 40 MiB build log to a single context window by asking for it.
(open! session id)Create (truncating) the log file for session/id and return the sink the
pump writes through: {:path :out}. Truncating is the point — a new spawn
under an id IS a new shell, and its log starts at offset 0.
Create (truncating) the log file for `session`/`id` and return the sink the
pump writes through: `{:path :out}`. Truncating is the point — a new spawn
under an id IS a new shell, and its log starts at offset 0.(read-chunk id file)(read-chunk id file {:keys [offset limit]})Read shell id's log file from :offset and answer a [[::log-chunk]].
With no :offset the read is the TAIL — the last default-chunk-bytes —
because that is what someone watching a live command wants, and the head is
one {:offset 0} away rather than gone. With an offset it reads FORWARD from
exactly that byte, which is what a loop feeding next-offset does.
A missing file is an empty chunk at offset 0, not an error: a shell that has printed nothing yet and a shell whose log was deleted read alike, and neither is worth an exception.
Read shell `id`'s log `file` from `:offset` and answer a [[::log-chunk]].
With no `:offset` the read is the TAIL — the last [[default-chunk-bytes]] —
because that is what someone watching a live command wants, and the head is
one `{:offset 0}` away rather than gone. With an offset it reads FORWARD from
exactly that byte, which is what a loop feeding `next-offset` does.
A missing file is an empty chunk at offset 0, not an error: a shell that has
printed nothing yet and a shell whose log was deleted read alike, and neither
is worth an exception.(session-dir session)Directory holding every shell log of ONE session.
Directory holding every shell log of ONE session.
(session-logs db-info session)Every indexed shell log of session, newest start first. The answer to
"what did that build print" a turn later, with no handle in hand.
Rows are wire-shaped: snake_case string keys, exactly as index! wrote
them.
Every indexed shell log of `session`, newest start first. The answer to "what did that build print" a turn later, with no handle in hand. Rows are wire-shaped: snake_case string keys, exactly as [[index!]] wrote them.
(tee in sink)Wrap in so every byte READ is also written to sink. The pump keeps its
character-level line splitting for the ring view while the file receives the
stream verbatim, so the two can never disagree about what the shell printed.
Wrap `in` so every byte READ is also written to `sink`. The pump keeps its character-level line splitting for the ring view while the file receives the stream verbatim, so the two can never disagree about what the shell printed.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |