com.blockether.vis.ext.channel-tui.virtual — com.blockether/vis-channel-tui 0.1.2

Virtualized chat-panel layout - the React-window pattern for our Lanterna scrollback.

── Why this exists ──────────────────────────────────────────────────────── The pre-virtualisation render path projected (i.e. ran format-answer-with-thinking over) EVERY assistant message on EVERY frame, even ones scrolled far off-screen. For finalized bubbles the outer format-answer-with-thinking LRU made each per-frame call ~315 ns once warm, but cold-opening a long session paid the full ~500 ms format cost per big trace bubble before the FIRST frame ever made it to the terminal. Session 954bf315 (2 x ~500ms bubbles -> ~870 ms cold paint) was the live trigger: the screen stayed bg-fill until both big bubbles formatted, which read as "frozen TUI on open".

── The shape of the fix ───────────────────────────────────────────────────

1. Cheap row-height ESTIMATE per message, derived from the trace shape (iteration count, code-form count, answer length) without touching the markdown tokenizer or the wrap-text engine.
2. layout plans the next paint: estimates every message's height, pins the auto-bottom scroll position, and - only for messages whose viewport interval is non-empty - runs the FULL projection
   • real bubble-height measurement. Off-screen bubbles never trigger format-answer-with-thinking.
3. The result carries the visible subset (with screen-row coordinates) plus a refined total-h and eff-scroll for scrollbar geometry. Callers paint visible directly.

── What this is NOT ───────────────────────────────────────────────────────

• Not a windowing library - we already have the bubble painter. This namespace is pure layout planning.
• Not a height-precision oracle - the estimates are intentionally rough (within ~2x). Off-screen accuracy only nudges the scrollbar thumb; visible bubbles always paint at their REAL height because they go through the full projection pass.
• Not a cache - the cache layers live in render.clj (format-answer-with-thinking, bubble-height, format-answer-markdown). This namespace just decides which messages get to consult those caches each frame.

── Caveat: scrollbar drift on first scroll up ───────────────────────────── When the user scrolls UP into a region whose estimates were wrong, the next layout will re-project those messages and refine their heights. The scrollbar thumb may shift one row as the total-h sum corrects. Mirrors react-window's VariableSizeList behaviour; eye-acceptable in practice.

(set! *warn-on-reflection* true) and *unchecked-math* :warn-on-boxed clean across this namespace per the repo's GraalVM ratchet.

estimated-height clj

(estimated-height message bubble-w)

(estimated-height message bubble-w detail-expansions session-id)

Cheap estimate of how many rows a message will paint at width bubble-w. Order of magnitude only - the full painter is authoritative for visible messages. NEVER calls wrap-text, markdown->lines, or format-answer-with-thinking.

INVARIANT: estimates must OVERSHOOT. When the real height replaces an estimate mid-scroll, total-h may shrink but never grow; growth moves the thumb TOWARD the bottom while the user scrolls UP — the visible scrollbar jump. So every section is sized as ceil of its folded rows (wrapped-rows-est) against a slightly NARROWER width than the painter's, and collapsed-by-default sections (thinking peek, result preview) are capped at the painter's preview limit + chrome ONLY when nothing in the message is expanded.

Shape mirrored from format-iteration-entry-entries:

• code — soft-folded at the bubble edge (p/fold-cols), painted in full: fold rows + 2 pad rows.
• result — markdown at fill-w, collapsed to a reasoning-preview-line-limit peek + +N more row (tool cards collapse to a headline — the peek cap over-estimates those by a few rows, which is fine).
• thinking — ALWAYS behind the ▸ THINKING accordion: peek rows + ~5 band-chrome rows; full height when expanded.
• answer — folded at min(60, fill-w): markdown chrome (headings, fences, list gaps) makes /60 the safe historical ballpark at any width ≥ 60, and narrower terminals fold at their own width. User / plain-assistant text goes through the markdown walker, which word-wraps and inserts block chrome — fold at 3/4 width to stay above it.

height-cache-size clj

(height-cache-size)

Current sticky-height entry count (handy for tests / diagnostics).

invalidate-heights! clj

(invalidate-heights!)

Drop the sticky height cache AND the estimate memo. Tests + whole-cache busts (registry-toggle resync) call this.

layout clj

(layout messages
        bubble-w
        settings
        scroll
        inner-h
        {:keys [progress loading? progress-extra]
         :or {progress nil loading? false}}
        &
        [{:keys [session-id detail-expansions prev-offsets]}])

Plan a paint of messages into a vertical viewport of inner-h rows at width bubble-w. Returns:

{:total-h <long> ;; sum of (real-or-estimated) heights :eff-scroll <long> ;; clamped scroll offset :heights <vec long> ;; one per message; real for visible, est for off-screen :offsets <vec long> ;; cumulative running sum, one entry longer than messages :visible [{:idx N :top R :height H :projected M} ...]}

scroll is nil for auto-bottom (jump to the latest message) or a non-negative long for a specific row offset.

loading? swaps the LAST assistant message's :text to the live spinner-led progress block (render/progress->text) only when that bubble intersects the viewport. Auto-bottom keeps live progress visible; manual scrollback does not re-project off-screen live progress.

The visible set is computed in TWO passes:

1. Estimate every height -> cumulative offsets -> clamp scroll -> identify candidate visible idxs.
2. Project + measure ONLY those candidates. Refine :heights, recompute :offsets, re-clamp :eff-scroll, recompute the visible interval (a bubble whose real height is much smaller than its estimate may have neighbours that NOW intersect the viewport - so we widen by one on each side as a cheap no-extra-projection fallback if you want strict pixel-perfect scrolling, run layout twice; in practice one pass is fine because the height delta is bounded by the heuristic).

No I/O, no Lanterna, no draw side effects. Pure planning.

NOTE on the signature: Clojure caps primitive-hinted fns at 4 args, and this one takes 6, so bubble-w / inner-h are plain Object args here - we cast with long inside the body.

pre-warm! clj

(pre-warm! messages
           bubble-w
           settings
           &
           [{:keys [session-id detail-expansions on-warm]}])

Spawn a daemon worker thread that calls project-message and bubble-height for every message in messages so the LRU is hot by the time the user scrolls. Walks in REVERSE order - the user almost always scrolls UP from the bottom, so the next-to- last message warms first, the FIRST message last.

Returns the Thread so the caller can stop it on session switch / shutdown via stop-pre-warm!. Returns nil for an empty session (nothing to warm).

Settings parity with layout: pass the SAME settings map so the cached entries match the keys subsequent layout passes will look up. Mismatched settings = wasted compute (the layout pass will format again with different cache keys).

:on-warm (0-arg fn, optional) fires on the worker thread as heights land in the sticky cache — periodically (every warm-bump-every bubbles) AND once when the whole session is warmed. Callers wire this to a render-version bump so total-h SETTLES to its fully-measured value while the user is still idle at auto-bottom (where the thumb is pinned to the bottom, so the settle is invisible) instead of snapping in one ~20% step on the first wheel-up. Without it the background warm populates the cache silently and the correction is deferred — and applied all at once — to the next render the user happens to trigger by scrolling. See the :caveat note at the top of this ns.

pre-warm-recent! clj

(pre-warm-recent! messages bubble-w settings)

(pre-warm-recent! messages
                  bubble-w
                  settings
                  {:keys [session-id detail-expansions] :as opts})

Synchronously warm the RECENT tail of a session before the background worker kicks in.

Why: pre-warm! is async by design (fast startup), but if the user wheel-scrolls immediately after opening a heavy session, they can still hit a cold big-trace bubble before the daemon gets there. Warming the newest tail on the caller thread eliminates that first-scroll cliff while still keeping the full-history warm async.

Options:

• :count max number of newest messages to warm (default 16)
• :budget-ms wall-clock budget for this sync pass (default 120)

Returns the number of warmed messages. Safe on empty input.

project-message clj

(project-message message bubble-w settings)

(project-message message
                 bubble-w
                 settings
                 {:keys [session-id detail-expansions tail-lines window-start
                         window-num window-total-h]})

Apply the same :text projection screen/apply-settings used to apply, but for ONE message at a time so the virtual layer can call it lazily per-bubble. Hits the same caches apply-settings did.

:tail-lines opt (when present, positive long) routes the IR walker through ir-tui/ir->lines-tail so only the LAST tail-lines styled lines are produced - O(visible-tail) instead of O(body). Used by layout for the auto-scrolled tail-pinned bubble where the user only sees the bottom of the message. See A3 in autoresearch.

rewarm! clj

(rewarm! messages bubble-w settings opts)

Stop the in-flight background warm (if any) and start warming messages at bubble-w. Owns ONE worker thread process-wide - callers never juggle the Thread themselves. opts is passed to pre-warm! verbatim (:session-id, :detail-expansions, :on-warm). Returns the new worker Thread, or nil for an empty history.

The previous worker is interrupted fire-and-forget (join 0): callers sit on the input/event thread, and a stale worker finishing one extra bubble writes a VALID cache entry - unlike the invalidate-then-stop test flow stop-pre-warm!'s docstring worries about.

stop-pre-warm! clj

(stop-pre-warm! t)

(stop-pre-warm! t join-ms)

Interrupt a pre-warm thread previously returned by pre-warm! and wait briefly for it to acknowledge the interrupt. Safe on nil and on already-finished threads.

Why join instead of fire-and-forget: the daemon checks (.isInterrupted …) BETWEEN warm-message-height! calls, not inside one. A fire-and-forget interrupt lets the daemon finish its in-flight bubble and write one more cache entry AFTER this returns, which (a) flakes the next test that just called invalidate-heights! and (b) in production silently leaks one cache entry for the previous session across a session switch. Joining with a tight budget bounds both: a typical interrupt lands within microseconds; the 200 ms ceiling covers the worst-case bubble already mid-format.

Optional second arg overrides the join budget (ms). Pass 0 for the old fire-and-forget behavior.

stop-rewarm! clj

(stop-rewarm!)

Interrupt + briefly join the managed re-warm worker (shutdown / session close). Safe when none is running.