Foundation's contribution to the host vis doctor aggregator. ONE fn
(doctor-fn) returns the full message stream from two logical
sections, each stamping its own :check-id so the formatter
groups them under the same banner the original four-checks-vec
shape produced (plan §1 Q18 / §10):
::agents-md AGENTS.md presence / source / size; one :info line when found, one :warn line when neither AGENTS.md nor CLAUDE.md exists (rules silently absent is worth flagging even though it isn't an error per se).
These section fns are pure data -> message-seq; they don't mutate
anything and don't depend on the runtime environment beyond
what's needed to read the existing scanners. Activation
contract per plan: every registered extension's :ext/doctor-fn
runs regardless of :ext/activation-fn, so the section fns must
NOT assume :db-info or other env keys are present.
Foundation's contribution to the host `vis doctor` aggregator. ONE fn
(`doctor-fn`) returns the full message stream from two logical
sections, each stamping its own `:check-id` so the formatter
groups them under the same banner the original four-checks-vec
shape produced (plan §1 Q18 / §10):
::agents-md AGENTS.md presence / source / size; one
:info line when found, one :warn line
when neither AGENTS.md nor CLAUDE.md exists
(rules silently absent is worth flagging
even though it isn't an error per se).
These section fns are pure data -> message-seq; they don't mutate
anything and don't depend on the runtime environment beyond
what's needed to read the existing scanners. Activation
contract per plan: every registered extension's `:ext/doctor-fn`
runs regardless of `:ext/activation-fn`, so the section fns must
NOT assume `:db-info` or other env keys are present.Filesystem tools exposed as bare symbols in the Python sandbox.
Two layers:
Structured helpers for read / tree / search:
(cat path) ; -> {:path :anchors {<N:hash> text…} :next-offset N? :truncated? B}
(cat path n) ; first n lines from line 1
(cat path offset n) ; n lines starting at line offset (1-based)
(cat path :tail) ; last 400 lines (tail)
(cat path :tail n) ; last n lines
(cat dir) ; a DIRECTORY path -> shallow listing {:path :entries [{name path type size}] :depth}
(cat dir opts) ; opts keys: depth (recurse) / is_hidden / is_respect_gitignore
(grep query) ; -> content hits (anchored) + ranked file-NAME matches;
; query = a term or list of terms (OR), smart-case
; substring. Opts: paths/include/limit/is_hidden
Cwd-safe wrappers over the babashka.fs file API. patch is
the canonical text edit surface:
(cat path) (patch [edit-map]) ; keys: path / from_anchor / replace (create-dirs path) (copy src dest) (move src dest) (delete path) (delete-if-exists path) (exists? path)
Hard guard: every path must stay inside the session's working
directory (fs/cwd); .. traversal is rejected before any I/O.
Filesystem tools exposed as bare symbols in the Python sandbox.
Two layers:
1. Structured helpers for read / tree / search:
(cat path) ; -> {:path :anchors {<N:hash> text…} :next-offset N? :truncated? B}
(cat path n) ; first n lines from line 1
(cat path offset n) ; n lines starting at line `offset` (1-based)
(cat path :tail) ; last 400 lines (tail)
(cat path :tail n) ; last n lines
(cat dir) ; a DIRECTORY path -> shallow listing {:path :entries [{name path type size}] :depth}
(cat dir opts) ; opts keys: depth (recurse) / is_hidden / is_respect_gitignore
(grep query) ; -> content hits (anchored) + ranked file-NAME matches;
; query = a term or list of terms (OR), smart-case
; substring. Opts: paths/include/limit/is_hidden
2. Cwd-safe wrappers over the babashka.fs file API. `patch` is
the canonical text edit surface:
(cat path)
(patch [edit-map]) ; keys: path / from_anchor / replace
(create-dirs path)
(copy src dest)
(move src dest)
(delete path)
(delete-if-exists path)
(exists? path)
Hard guard: every path must stay inside the session's working
directory (`fs/cwd`); `..` traversal is rejected before any I/O.Structural INDEX: a high-level, line-ranged skeleton of a source file produced via tree-sitter (com.blockether/tree-sitter-language-pack, which sources Clojure from our own grammar fork).
Every item carries FULL anchors — the same <lineno>:<hash> anchors cat
emits and patch consumes — for its first and last line, so you can replace
a whole definition straight from the index with one
patch([{path P, from_anchor start, to_anchor end, replace …}]), no intermediate
cat. Each line is:
<kind> <name> <signature> @<start-anchor>..<end-anchor>
e.g.
class Greeter @3:a1b..7:c2d function hello @6:1b2..7:c2d function main @9:3c4..10:5d6
Read it BEFORE cat: a cheap map of a file so you jump straight to the right
range (and its anchors) instead of reading the whole file.
Requiring this namespace also requires the native resolver, which selects the right per-platform FFI library at runtime.
Structural INDEX: a high-level, line-ranged skeleton of a source file
produced via tree-sitter (com.blockether/tree-sitter-language-pack, which
sources Clojure from our own grammar fork).
Every item carries FULL anchors — the same `<lineno>:<hash>` anchors `cat`
emits and `patch` consumes — for its first and last line, so you can replace
a whole definition straight from the index with one
`patch([{path P, from_anchor start, to_anchor end, replace …}])`, no intermediate
`cat`. Each line is:
<kind> <name> <signature> @<start-anchor>..<end-anchor>
e.g.
class Greeter @3:a1b..7:c2d
function hello @6:1b2..7:c2d
function main @9:3c4..10:5d6
Read it BEFORE `cat`: a cheap map of a file so you jump straight to the right
range (and its anchors) instead of reading the whole file.
Requiring this namespace also requires the native resolver, which selects the
right per-platform FFI library at runtime.Pure hashline primitives for content-addressed reading and editing.
Public surface:
split-content-lines / char-offset-at-line
line-hash / lines->anchors text -> lineno:hash / {ln anchor}
render-hashline-block / -range-block tuples -> <hash>| text gutter
indices-matching-hash / resolve-anchor-edit-span self-locating range replace
Pure hashline primitives for content-addressed reading and editing.
Public surface:
split-content-lines / char-offset-at-line
line-hash / lines->anchors text -> lineno:hash / {ln anchor}
render-hashline-block / -range-block tuples -> `<hash>| text` gutter
indices-matching-hash / resolve-anchor-edit-span self-locating range replaceThin Clojure adapter over the pack's Java structural-edit engine
(dev.kreuzberg.treesitterlanguagepack.StructuralApi). All the work —
locate the definition by name from the tree-sitter outline, splice its line
span, and re-parse to refuse syntax-breaking edits — lives in Java so it is
language-neutral, reusable from any JVM consumer, and native-image clean.
This namespace only maps vis op keywords onto the Java API.
Thin Clojure adapter over the pack's Java structural-edit engine (`dev.kreuzberg.treesitterlanguagepack.StructuralApi`). All the work — locate the definition by name from the tree-sitter outline, splice its line span, and re-parse to refuse syntax-breaking edits — lives in Java so it is language-neutral, reusable from any JVM consumer, and native-image clean. This namespace only maps vis op keywords onto the Java API.
Language-neutral STRUCTURAL ZIPPER over the tree-sitter pack (306+ langs) —
the unified cursor the name-based structural ops were missing.
A node's location is a STATELESS PATH: a vector of NAMED-child indices from
the root (e.g. [2 0] = first named child of the third named child of the
file). Stateless means it round-trips cleanly through async tool calls — no
live native cursor to keep between calls. Relative moves (down/up/next/prev)
are pure path arithmetic on top, so the model navigates like a rewrite-clj
zipper but over EVERY language tree-sitter understands.
Edits splice the target node's UTF-8 byte range and RE-PARSE, refusing a
result that introduces a syntax error the original didn't have — the same
safety contract as structural. Pairs with the name-based ops: locate a def
by name, then walk into it by path.
All native handles (Parser/Tree/Node) are opened and closed inside each call; only plain Clojure data escapes.
Language-neutral STRUCTURAL ZIPPER over the tree-sitter pack (306+ langs) — the unified cursor the name-based `structural` ops were missing. A node's location is a STATELESS PATH: a vector of NAMED-child indices from the root (e.g. `[2 0]` = first named child of the third named child of the file). Stateless means it round-trips cleanly through async tool calls — no live native cursor to keep between calls. Relative moves (down/up/next/prev) are pure path arithmetic on top, so the model navigates like a rewrite-clj zipper but over EVERY language tree-sitter understands. Edits splice the target node's UTF-8 byte range and RE-PARSE, refusing a result that introduces a syntax error the original didn't have — the same safety contract as `structural`. Pairs with the name-based ops: locate a def by name, then walk into it by path. All native handles (Parser/Tree/Node) are opened and closed inside each call; only plain Clojure data escapes.
Thin re-export shim — project-guidance discovery moved to
com.blockether.vis.internal.agents. AGENTS.md / CLAUDE.md
handling is core functionality (drives system prompt + ctx
digest), so it now lives outside the foundation extension.
This shim keeps the legacy main-agent-instructions tool +
any downstream (:project ctx) :guidance consumers compiling
while they migrate to the internal namespace.
Thin re-export shim — project-guidance discovery moved to `com.blockether.vis.internal.agents`. AGENTS.md / CLAUDE.md handling is core functionality (drives system prompt + ctx digest), so it now lives outside the foundation extension. This shim keeps the legacy `main-agent-instructions` tool + any downstream `(:project ctx) :guidance` consumers compiling while they migrate to the internal namespace.
vis-foundation — the agent's environment-awareness layer.
Owns the environment facts: cwd, user, platform, shell, plus:
Model-facing VCS/workspace truth lives in session["workspace"].
Remaining helpers cover coarse project shape (languages,
monorepo, repositories) and cache invalidation (refresh!).
Runtime facts are computed lazily on first access and cached per
working-directory. The cache is invalidated automatically when
cwd changes between calls and explicitly by (refresh!).
vis-foundation — the agent's environment-awareness layer.
Owns the environment facts: cwd, user, platform, shell, plus:
* git repository facts via the git binary (root, branch, dirty status,
submodules, worktree),
* a bounded language scan over the working tree (top languages
by file count, primary language),
* monorepo / multi-package shape detection (polylith, workspace,
submodules) by counting per-ecosystem manifests.
Model-facing VCS/workspace truth lives in `session["workspace"]`.
Remaining helpers cover coarse project shape (`languages`,
`monorepo`, `repositories`) and cache invalidation (`refresh!`).
Runtime facts are computed lazily on first access and cached per
working-directory. The cache is invalidated automatically when
`cwd` changes between calls and explicitly by `(refresh!)`.Git introspection for the environment block, backed by the native git
binary (via internal.git).
Returns a snapshot map for the repository that contains start
(typically the JVM working directory). nil when start is not inside any
git repository. Never throws — every git call is guarded; on any failure we
degrade gracefully to nil or a reduced-shape map.
The expensive call is git status (working-tree walk). When it fails or is
suppressed, the snapshot drops the dirty-status fields instead of stalling
the system-prompt build.
Git introspection for the environment block, backed by the native `git` binary (via `internal.git`). Returns a snapshot map for the repository that contains `start` (typically the JVM working directory). nil when `start` is not inside any git repository. Never throws — every git call is guarded; on any failure we degrade gracefully to nil or a reduced-shape map. The expensive call is `git status` (working-tree walk). When it fails or is suppressed, the snapshot drops the dirty-status fields instead of stalling the system-prompt build.
Host-side facts read from JDK system properties and process environment variables.
No I/O, no shell-out, no third-party deps. Cheap to compute, safe to call from any thread, never throws.
Host-side facts read from JDK system properties and process environment variables. No I/O, no shell-out, no third-party deps. Cheap to compute, safe to call from any thread, never throws.
Bounded language scan over a directory tree.
Walks the tree with Files/walkFileTree, skipping common
non-source subdirectories (.git, node_modules, target, ...)
via FileVisitResult/SKIP_SUBTREE. Counts files and bytes per
language using a small extension-to-language map.
The walk has TWO hard guards: a max-file count and a wall-time
deadline. Either one stops the walk via TERMINATE. Callers get
a possibly-partial result; on a small repo the result is exact.
No third-party deps. Reflection-clean.
Bounded language scan over a directory tree. Walks the tree with `Files/walkFileTree`, skipping common non-source subdirectories (`.git`, `node_modules`, `target`, ...) via `FileVisitResult/SKIP_SUBTREE`. Counts files and bytes per language using a small extension-to-language map. The walk has TWO hard guards: a max-file count and a wall-time deadline. Either one stops the walk via `TERMINATE`. Callers get a possibly-partial result; on a small repo the result is exact. No third-party deps. Reflection-clean.
Monorepo / multi-package detection.
Walks the tree once (bounded), counting per-language manifest files at any depth below the root. >=2 manifests of the same kind in distinct subdirectories signals a multi-package workspace; we report the kind, the count, and a best-guess shape label (e.g. "polylith", "workspace", "submodules").
Reflection-clean. Honors the same skip-directory list as the language scanner.
Monorepo / multi-package detection. Walks the tree once (bounded), counting per-language manifest files at any depth below the root. >=2 manifests of the same kind in distinct subdirectories signals a multi-package workspace; we report the kind, the count, and a best-guess shape label (e.g. "polylith", "workspace", "submodules"). Reflection-clean. Honors the same skip-directory list as the language scanner.
Build compact foundation environment data for ctx. No prompt labels.
Build compact foundation environment data for `ctx`. No prompt labels.
Bounded discovery of multiple Git repositories below the current
project root. This catches multirepo workspaces where the user's cwd
is a parent directory or a primary repo that vendors sibling/nested
repos outside .gitmodules.
Returns compact per-repo Git summaries for the system prompt. Full
status walks are bounded per repo by git/snapshot; the repository
scan itself is bounded by max files, max repos, and a wall-clock
deadline. Never throws.
Bounded discovery of multiple Git repositories below the current project root. This catches multirepo workspaces where the user's cwd is a parent directory or a primary repo that vendors sibling/nested repos outside `.gitmodules`. Returns compact per-repo Git summaries for the system prompt. Full status walks are bounded per repo by `git/snapshot`; the repository scan itself is bounded by max files, max repos, and a wall-clock deadline. Never throws.
The single git tool — a thin, honest proxy to the host git binary.
ONE built-in Python function, git, runs git <args…> in the active
workspace root and returns a TOTAL, string-keyed result the model reads
directly: {"cmd", "args", "stdout", "stderr", "exit", "duration_ms", "timed_out", "timeout_secs"} — every key ALWAYS present (stderr "" when
empty, exit None only when the run timed out), so no field ever KeyErrors.
A non-zero exit is DATA, not a tool failure — the model reads it like it
would in a terminal.
This REPLACES the old JGit-backed git_* surface (foundation-git): no
embedded git implementation, no SSH/BouncyCastle stack — the only git is
the one already on the user's PATH, so behaviour matches their shell
exactly. Read-only workspace facts (branch/dirty/ahead-behind for the
footer, env block, file picker) still flow through
com.blockether.vis.internal.git; this namespace is purely the model-
facing command tool.
Built-in (bare git in the sandbox, next to cat/rg), gated to
activate only when the workspace sits inside a repository.
The single `git` tool — a thin, honest proxy to the host `git` binary.
ONE built-in Python function, `git`, runs `git <args…>` in the active
workspace root and returns a TOTAL, string-keyed result the model reads
directly: `{"cmd", "args", "stdout", "stderr", "exit", "duration_ms",
"timed_out", "timeout_secs"}` — every key ALWAYS present (`stderr` "" when
empty, `exit` None only when the run timed out), so no field ever KeyErrors.
A non-zero exit is DATA, not a tool failure — the model reads it like it
would in a terminal.
This REPLACES the old JGit-backed `git_*` surface (foundation-git): no
embedded git implementation, no SSH/BouncyCastle stack — the only git is
the one already on the user's PATH, so behaviour matches their shell
exactly. Read-only workspace facts (branch/dirty/ahead-behind for the
footer, env block, file picker) still flow through
`com.blockether.vis.internal.git`; this namespace is purely the model-
facing command tool.
Built-in (bare `git` in the sandbox, next to `cat`/`rg`), gated to
activate only when the workspace sits inside a repository.harness compatibility layer — a BUILT-IN foundation module (ships in the
main jar, always present, gated by toggles) that exposes the AGENTS and
SKILLS vis' own project
dir and other AI coding HARNESSES (Claude Code, pi, opencode, the agents
standard, …) leave on disk to the vis model. The sibling of the shell
layer's POSIX compat. Vis reads its OWN project-local skills from
.vis/skills (highest precedence).
Two bare verbs (bound like cat/rg via :ext.engine/builtin? true):
SKILLS are PROGRESSIVE: the prompt lists every skill name — description
(cheap — always present), and
skill(name) loads the FULL SKILL.md + its bundled resource PATHS on
demand so the model spends tokens only on the one it uses.
AGENTS are an ALIAS to sub_loop: agent(name, prompt) runs the named
agent as a CHILD loop whose system prompt IS that agent's markdown body,
on its declared model.
Both bare verbs are unconditionally available — skills/agents/commands have no user toggle; the layer is always active.
`harness` compatibility layer — a BUILT-IN foundation module (ships in the main jar, always present, gated by toggles) that exposes the AGENTS and SKILLS vis' own project dir and other AI coding HARNESSES (Claude Code, pi, opencode, the agents standard, …) leave on disk to the vis model. The sibling of the shell layer's POSIX compat. Vis reads its OWN project-local skills from `.vis/skills` (highest precedence). Two bare verbs (bound like cat/rg via `:ext.engine/builtin? true`): - SKILLS are PROGRESSIVE: the prompt lists every skill `name — description` (cheap — always present), and `skill(name)` loads the FULL `SKILL.md` + its bundled resource PATHS on demand so the model spends tokens only on the one it uses. - AGENTS are an ALIAS to `sub_loop`: `agent(name, prompt)` runs the named agent as a CHILD loop whose system prompt IS that agent's markdown body, on its declared model. Both bare verbs are unconditionally available — skills/agents/commands have no user toggle; the layer is always active.
Cross-HARNESS discovery of agents + skills — the sibling of the shell layer's POSIX compat, for the agent/skill definitions vis' OWN project dir and OTHER AI coding harnesses (Claude Code, pi, opencode, the agents standard, …) leave on disk.
An AGENT is a markdown file with YAML-ish --- frontmatter
(name, description, model, tools) + a body that IS a system
prompt. A SKILL is a SKILL.md (same frontmatter, name+description) in
its own directory, alongside bundled resource files.
Discovery is PURE except for the directory scan: parse-frontmatter,
parse-agent, parse-skill-meta, and dedup-by-name take strings and
are unit-tested without the filesystem; the discover-* fns walk the
known source roots. Precedence is source ORDER, first-name-wins
(vis project-local > other harnesses' project > user > plugin; Vis and
Claude before pi/agents/opencode).
Cross-HARNESS discovery of agents + skills — the sibling of the shell layer's POSIX compat, for the agent/skill definitions vis' OWN project dir and OTHER AI coding harnesses (Claude Code, pi, opencode, the agents standard, …) leave on disk. An AGENT is a markdown file with YAML-ish `---` frontmatter (`name`, `description`, `model`, `tools`) + a body that IS a system prompt. A SKILL is a `SKILL.md` (same frontmatter, name+description) in its own directory, alongside bundled resource files. Discovery is PURE except for the directory scan: `parse-frontmatter`, `parse-agent`, `parse-skill-meta`, and `dedup-by-name` take strings and are unit-tested without the filesystem; the `discover-*` fns walk the known source roots. Precedence is source ORDER, first-name-wins (vis project-local > other harnesses' project > user > plugin; Vis and Claude before pi/agents/opencode).
Programmatic introspection of the agent's own state from inside
:code. The public state surface is deliberately small:
(session-state [session-id]) -> data map, including raw LLM diagnostics(session-report-html [session-id]) -> HTML report rendered from that dataEverything else in this namespace is implementation detail. The agent
gets the data once, manipulates it via plain Clojure (get-in,
filter, map, etc.), and renders the same data to Markdown only
when presentation is needed.
Every function is a pure read off the same DB tables the projection layer reads from (or a classpath read for the doc accessors). Failures return nil/[], never throw, so a misbehaving introspection call cannot break iteration execution.
Opt-in: not auto-loaded by default. Add this jar to the classpath to enable.
Programmatic introspection of the agent's own state from inside `:code`. The public state surface is deliberately small: - `(session-state [session-id])` -> data map, including raw LLM diagnostics - `(session-report-html [session-id])` -> HTML report rendered from that data Everything else in this namespace is implementation detail. The agent gets the data once, manipulates it via plain Clojure (`get-in`, `filter`, `map`, etc.), and renders the same data to Markdown only when presentation is needed. Every function is a pure read off the same DB tables the projection layer reads from (or a classpath read for the doc accessors). Failures return nil/[], never throw, so a misbehaving introspection call cannot break iteration execution. Opt-in: not auto-loaded by default. Add this jar to the classpath to enable.
Language-neutral FORMAT / TEST / REPL_EVAL / START_REPL dispatch.
Language extensions register handlers under :ext/language-tools; this
foundation surface exposes stable bare tool names and dispatches to the
active handler for the requested/current language. REPL lifecycle is resource
backed: repl creates a language-owned session resource and repl_stop
stops one by id. Live REPLs also surface in the ctx resources block.
Language-neutral FORMAT / TEST / REPL_EVAL / START_REPL dispatch. Language extensions register handlers under `:ext/language-tools`; this foundation surface exposes stable bare tool names and dispatches to the active handler for the requested/current language. REPL lifecycle is resource backed: `repl` creates a language-owned session resource and `repl_stop` stops one by id. Live REPLs also surface in the ctx `resources` block.
Minimal Model Context Protocol (MCP) client. Speaks JSON-RPC 2.0 over two transports:
:stdio — spawn the server process and frame newline-delimited JSON-RPC on its stdin/stdout (the dominant local-server pattern). A daemon thread drains stderr into the vis log so a chatty server never deadlocks on a full pipe.
:http — Streamable HTTP: POST each JSON-RPC message to one endpoint; the
reply is either application/json (one response) or
text/event-stream (SSE) — both handled. The Mcp-Session-Id
handed back by initialize rides on every later request, a
DELETE frees it on shutdown, and an optional GET listen loop
reacts to server-pushed notifications/tools/list_changed.
OAuth 2.1 (spec 2025-06-18) is supported for HTTP transports via
oauth.clj: pass :bearer-fn (a 0/1-arg fn yielding the current Bearer
token, called with the just-rejected token on 401). A 401 triggers a
single-flight refresh and one automatic retry.
A conn is a plain map of closures + state; the extension treats it
opaquely. Lifecycle: connect (which performs the initialize handshake) →
list-tools / call-tool → close.
Minimal Model Context Protocol (MCP) client. Speaks JSON-RPC 2.0 over two
transports:
:stdio — spawn the server process and frame newline-delimited JSON-RPC on
its stdin/stdout (the dominant local-server pattern). A daemon
thread drains stderr into the vis log so a chatty server never
deadlocks on a full pipe.
:http — Streamable HTTP: POST each JSON-RPC message to one endpoint; the
reply is either `application/json` (one response) or
`text/event-stream` (SSE) — both handled. The `Mcp-Session-Id`
handed back by `initialize` rides on every later request, a
`DELETE` frees it on shutdown, and an optional GET listen loop
reacts to server-pushed `notifications/tools/list_changed`.
OAuth 2.1 (spec `2025-06-18`) is supported for HTTP transports via
`oauth.clj`: pass `:bearer-fn` (a 0/1-arg fn yielding the current Bearer
token, called with the just-rejected token on 401). A 401 triggers a
single-flight refresh and one automatic retry.
A `conn` is a plain map of closures + state; the extension treats it
opaquely. Lifecycle: `connect` (which performs the `initialize` handshake) →
`list-tools` / `call-tool` → `close`.Built-in Model Context Protocol (MCP) surface. The gateway daemon owns ONE
shared pool of MCP connections ({server {:conn spec}}); every session sees
the same live tools. Always on: MCP is core infrastructure, not a droppable
plug-in and not gated by any toggle. The pool is empty (and costs nothing)
until at least one server is declared in config.
Servers are declared natively in ~/.vis/state.yml:
{:mcp {:servers {"filesystem" {:transport :stdio :command "npx" :args ["-y" "@modelcontextprotocol/server-filesystem" "/path"]} "remote" {:transport :http :url "https://..." :headers {"Authorization" "Bearer ${MY_TOKEN}"} :timeout_ms 60000 :listen true :auth {:client_id "..." :scope "..."}} "stale" {:enabled false :url "..."}}}}
Every string in :headers / :env / :args / :url / :command / :cwd
supports ${ENV_VAR} interpolation from the host environment. :enabled false skips the server without deleting the entry. HTTP servers with no
static bearer transparently negotiate OAuth 2.1 on first 401 (RFC 9728
discovery + RFC 7591 dynamic client registration + PKCE loopback).
Five model-facing verbs under alias mcp (flat sandbox renders alias_name):
mcp__servers() — configured servers + status + tool counts
mcp__tools(server) — a server's tools (auto-connects)
mcp__call(server, tool, args) — call a tool (auto-connects)
mcp__connect(server) / mcp__disconnect(server) — manage the connection
Live connections + tool counts also ride in ctx under env.mcp.
Built-in Model Context Protocol (MCP) surface. The gateway daemon owns ONE
shared pool of MCP connections (`{server {:conn spec}}`); every session sees
the same live tools. Always on: MCP is core infrastructure, not a droppable
plug-in and not gated by any toggle. The pool is empty (and costs nothing)
until at least one server is declared in config.
Servers are declared natively in `~/.vis/state.yml`:
{:mcp {:servers {"filesystem" {:transport :stdio :command "npx"
:args ["-y" "@modelcontextprotocol/server-filesystem" "/path"]}
"remote" {:transport :http :url "https://..."
:headers {"Authorization" "Bearer ${MY_TOKEN}"}
:timeout_ms 60000
:listen true
:auth {:client_id "..." :scope "..."}}
"stale" {:enabled false :url "..."}}}}
Every string in `:headers` / `:env` / `:args` / `:url` / `:command` / `:cwd`
supports `${ENV_VAR}` interpolation from the host environment. `:enabled
false` skips the server without deleting the entry. HTTP servers with no
static bearer transparently negotiate OAuth 2.1 on first 401 (RFC 9728
discovery + RFC 7591 dynamic client registration + PKCE loopback).
Five model-facing verbs under alias `mcp` (flat sandbox renders `alias_name`):
mcp__servers() — configured servers + status + tool counts
mcp__tools(server) — a server's tools (auto-connects)
mcp__call(server, tool, args) — call a tool (auto-connects)
mcp__connect(server) / mcp__disconnect(server) — manage the connection
Live connections + tool counts also ride in ctx under `env.mcp`.One JDK java.net.http.HttpClient shared by every MCP subsystem
(request/response, OAuth discovery + token exchange, SSE listen loop).
Rationale — perf + threading:
sendAsync completions. Two client
instances = two selector pools = wasted daemon threads for zero win.
So we share ONE instance across client.clj + oauth.clj.newVirtualThreadPerTaskExecutor — on Java 25 virtual
threads no longer pin on synchronized (JEP 491), so blocking-.send
calls made from a virtual thread never park a carrier. Bounded footprint,
no cached thread-pool growth under bursty MCP load.defonce + delay = constructed lazily at runtime, never at
namespace-load; safe for GraalVM native-image (the client owns non-heap
state that can't land in the build image).One JDK `java.net.http.HttpClient` shared by every MCP subsystem
(request/response, OAuth discovery + token exchange, SSE listen loop).
Rationale — perf + threading:
* The JDK client owns a small internal selector pool AND an executor used
to run response-body pipelines and `sendAsync` completions. Two client
instances = two selector pools = wasted daemon threads for zero win.
So we share ONE instance across `client.clj` + `oauth.clj`.
* The executor is a `newVirtualThreadPerTaskExecutor` — on Java 25 virtual
threads no longer pin on `synchronized` (JEP 491), so blocking-`.send`
calls made from a virtual thread never park a carrier. Bounded footprint,
no cached thread-pool growth under bursty MCP load.
* `defonce` + `delay` = constructed lazily at runtime, never at
namespace-load; safe for GraalVM native-image (the client owns non-heap
state that can't land in the build image).OAuth 2.1 client for Model Context Protocol servers (spec 2025-06-18).
Flow — on HTTP 401 from an MCP server, we:
WWW-Authenticate: Bearer resource_metadata="..." (RFC 9728),
fall back to ${origin}/.well-known/oauth-protected-resource;authorization_servers[0];.well-known/oauth-authorization-server (RFC 8414) or
.well-known/openid-configuration for endpoints + capabilities;client_id;http://127.0.0.1:<ephemeral>/mcp-callback) — spawn a one-shot
com.sun.net.httpserver.HttpServer, print the URL, best-effort open
it in the user's browser, wait for ?code=;~/.vis/mcp-tokens/<server>.edn;com.blockether.vis.internal.oauth/make-file-refresher.The returned bearer-fn is a 0/1-arg function: 0-arg yields the current
bearer token (running the auth flow on first use); 1-arg (with the token the
server just rejected) forces a refresh.
OAuth 2.1 client for Model Context Protocol servers (spec `2025-06-18`).
Flow — on HTTP 401 from an MCP server, we:
1. read `WWW-Authenticate: Bearer resource_metadata="..."` (RFC 9728),
fall back to `${origin}/.well-known/oauth-protected-resource`;
2. GET the resource-metadata JSON → pick an `authorization_servers[0]`;
3. GET its `.well-known/oauth-authorization-server` (RFC 8414) or
`.well-known/openid-configuration` for endpoints + capabilities;
4. dynamic-client-register (RFC 7591) if the AS supports it, or use
the caller-supplied `client_id`;
5. run PKCE S256 authorization-code with a loopback redirect
(`http://127.0.0.1:<ephemeral>/mcp-callback`) — spawn a one-shot
`com.sun.net.httpserver.HttpServer`, print the URL, best-effort open
it in the user's browser, wait for `?code=`;
6. exchange code → access + refresh tokens; persist to
`~/.vis/mcp-tokens/<server>.edn`;
7. on later expiry / 401, refresh the token single-flight through
`com.blockether.vis.internal.oauth/make-file-refresher`.
The returned `bearer-fn` is a 0/1-arg function: 0-arg yields the current
bearer token (running the auth flow on first use); 1-arg (with the token the
server just rejected) forces a refresh.SINK for artifacts the sandbox PRODUCES — matplotlib figures (plt.show() /
plt.savefig()), anything a tool hands to vis_attach, AND anything written
into the per-context OUTBOX directory ($VIS_OUTBOX) — so the engine OWNS the
bytes (a session_iteration_attachment row) captured AT THE SOURCE, with NO
re-parsing of the model-facing stdout fence.
The old flow rendered each figure to a $TMPDIR/vis-mpl temp file, printed a
vis-image fence carrying just that PATH, then at persist time re-parsed the
fence out of stdout and re-read the (possibly already-gone) file. We control the
whole boundary, so that round-trip is gone: a producer renders/reads the bytes
HOST-side (matplotlib's __vis_mpl_render_file__ Java2D backend; vis_attach's
sandbox-confined open; the outbox filesystem tap in sandbox-fs) and, right
where it already holds the bytes, calls record-attachment! (or record-file!).
run-python-block binds *attachment-sink* to a fresh collector around each
block's eval and drains it into the block outcome's :attachments; the loop
stamps each with the block's tool-call-id and hands them to db-store-iteration!'s
:attachments. The stdout fence now serves ONLY the inline TUI/web display +
ASCII fallback — never persistence.
Deliberately dependency-free (no AWT, no vis.core): safe to require from BOTH
a render shim and the hot engine loop without dragging Java2D or a require
cycle.
SINK for artifacts the sandbox PRODUCES — matplotlib figures (`plt.show()` / `plt.savefig()`), anything a tool hands to `vis_attach`, AND anything written into the per-context OUTBOX directory (`$VIS_OUTBOX`) — so the engine OWNS the bytes (a `session_iteration_attachment` row) captured AT THE SOURCE, with NO re-parsing of the model-facing stdout fence. The old flow rendered each figure to a `$TMPDIR/vis-mpl` temp file, printed a `vis-image` fence carrying just that PATH, then at persist time re-parsed the fence out of stdout and re-read the (possibly already-gone) file. We control the whole boundary, so that round-trip is gone: a producer renders/reads the bytes HOST-side (matplotlib's `__vis_mpl_render_file__` Java2D backend; `vis_attach`'s sandbox-confined `open`; the outbox filesystem tap in `sandbox-fs`) and, right where it already holds the bytes, calls `record-attachment!` (or `record-file!`). `run-python-block` binds `*attachment-sink*` to a fresh collector around each block's eval and drains it into the block outcome's `:attachments`; the loop stamps each with the block's tool-call-id and hands them to `db-store-iteration!`'s `:attachments`. The stdout fence now serves ONLY the inline TUI/web display + ASCII fallback — never persistence. Deliberately dependency-free (no AWT, no `vis.core`): safe to require from BOTH a render shim and the hot engine loop without dragging Java2D or a require cycle.
Pure-Java pseudo-terminal for background shell children — NO JNA, NO extracted native helper,
NO external tmux. Everything is a java.lang.foreign (Panama FFM) downcall
into the platform libc, so it survives GraalVM native-image the same way the
rest of vis's FFM surface (fff / rift / ruff / tree-sitter) does.
Why FFM and not pty4j:
libpty that it extracts to
a temp dir at runtime — two native-image headaches (reflection metadata for
JNA + a resource-extraction dance for the .dylib/.so).forkpty then execve in the
child — does NOT work from a JVM: invoking an FFM downcall MethodHandle after
fork() is not async-signal-safe and SIGBUSes the child. (Verified.)posix_spawn sidesteps that entirely: libc does the fork+exec ATOMICALLY in
native code, so vis only ever issues ONE parent-side downcall and never runs
any JVM code in the child. Paired with openpty (master/slave fds) + a
dup2 of the slave onto the child's 0/1/2, the child gets a real TTY:
isatty() is true, $TERM is honoured, stdin is writable.Public surface — spawn! returns a PLAIN MAP (not a java.lang.Process; a
runtime proxy/gen-class would break native-image), shaped for the
internal.foundation.shell background pump:
{:pid <long> OS pid (a genuine child — ProcessHandle/of
works, unlike a pty4j process)
:in <java.io.InputStream> master-fd reader (a real piped stream)
:send (fn [^bytes b]) write bytes to the master (the stdin channel)
:wait (fn [] <int>) block until exit, reap, return the exit code
:alive? (fn [] <bool>)
:destroy (fn [force?]) SIGTERM (false) / SIGKILL (true) the child}
Pure-Java pseudo-terminal for background `shell` children — NO JNA, NO extracted native helper,
NO external `tmux`. Everything is a `java.lang.foreign` (Panama FFM) downcall
into the platform libc, so it survives GraalVM native-image the same way the
rest of vis's FFM surface (fff / rift / ruff / tree-sitter) does.
Why FFM and not pty4j:
- pty4j drags in JNA *and* ships its own compiled `libpty` that it extracts to
a temp dir at runtime — two native-image headaches (reflection metadata for
JNA + a resource-extraction dance for the .dylib/.so).
- The obvious pure-FFM shortcut — call libc `forkpty` then `execve` in the
child — does NOT work from a JVM: invoking an FFM downcall MethodHandle after
`fork()` is not async-signal-safe and SIGBUSes the child. (Verified.)
- `posix_spawn` sidesteps that entirely: libc does the fork+exec ATOMICALLY in
native code, so vis only ever issues ONE parent-side downcall and never runs
any JVM code in the child. Paired with `openpty` (master/slave fds) + a
`dup2` of the slave onto the child's 0/1/2, the child gets a real TTY:
`isatty()` is true, `$TERM` is honoured, stdin is writable.
Public surface — `spawn!` returns a PLAIN MAP (not a `java.lang.Process`; a
runtime `proxy`/`gen-class` would break native-image), shaped for the
`internal.foundation.shell` background pump:
{:pid <long> OS pid (a genuine child — `ProcessHandle/of`
works, unlike a pty4j process)
:in <java.io.InputStream> master-fd reader (a real piped stream)
:send (fn [^bytes b]) write bytes to the master (the stdin channel)
:wait (fn [] <int>) block until exit, reap, return the exit code
:alive? (fn [] <bool>)
:destroy (fn [force?]) SIGTERM (false) / SIGKILL (true) the child}Passthrough bridge on top of the FFM pseudo-terminal (internal.foundation.pty).
The problem it solves: a background shell child runs INSIDE the vis process (the FFM
PTY master fd + reader thread live in vis's heap). That's great for the agent
(the shell send / logs ops) but a HUMAN can't jump into the live terminal to
finish a step the agent can't — click through a browser OAuth, answer a prompt
only a person can. tmux gets that for free because its server is a separate
daemon you can tmux attach to; the FFM child is not.
This namespace restores that capability WITHOUT tmux: each background PTY
optionally exposes a per-shell UNIX-DOMAIN SOCKET. vis is the server (it holds
the master fd); vis extension shell attach <id> is a thin client the human runs in
their OWN Terminal.app. On connect the server (a) tees live master output to the
socket and (b) forwards the socket's bytes to the master (stdin) — a genuine
bidirectional passthrough. Multiple humans can attach at once; detaching just
drops the socket and leaves the child running (exactly like tmux detach).
Everything here is stdlib: java.nio.channels AF_UNIX sockets (JDK 16+,
already in vis's native-image reachability metadata) on the server side, and
stty for raw mode on the client side (the human's interactive shell always
has it). No JNA, no new dep, native-image clean.
Passthrough bridge on top of the FFM pseudo-terminal (internal.foundation.pty). The problem it solves: a background `shell` child runs INSIDE the vis process (the FFM PTY master fd + reader thread live in vis's heap). That's great for the agent (the shell send / logs ops) but a HUMAN can't jump into the live terminal to finish a step the agent can't — click through a browser OAuth, answer a prompt only a person can. tmux gets that for free because its server is a separate daemon you can `tmux attach` to; the FFM child is not. This namespace restores that capability WITHOUT tmux: each background PTY optionally exposes a per-shell UNIX-DOMAIN SOCKET. vis is the server (it holds the master fd); `vis extension shell attach <id>` is a thin client the human runs in their OWN Terminal.app. On connect the server (a) tees live master output to the socket and (b) forwards the socket's bytes to the master (stdin) — a genuine bidirectional passthrough. Multiple humans can attach at once; detaching just drops the socket and leaves the child running (exactly like `tmux detach`). Everything here is stdlib: `java.nio.channels` AF_UNIX sockets (JDK 16+, already in vis's native-image reachability metadata) on the server side, and `stty` for raw mode on the client side (the human's interactive shell always has it). No JNA, no new dep, native-image clean.
Vis self-documentation lookup — the vis_docs sandbox tool.
Vis ships its documentation as embedded markdown pages under
vis-docs/ on the classpath (the same corpus the website and the
gateway /docs site render, discovered via each artifact's
vis-docs/vis-docs.edn manifest — so extensions' doc pages are
lookup-able too). This namespace exposes that corpus to the MODEL
through one observation tool so vis can answer questions about
ITSELF — features, configuration, how to write an extension — from
its real docs instead of guessing.
Progressive disclosure: CORE routes vis questions here; page content is only paid for when actually fetched.
Vis self-documentation lookup — the `vis_docs` sandbox tool. Vis ships its documentation as embedded markdown pages under `vis-docs/` on the classpath (the same corpus the website and the gateway `/docs` site render, discovered via each artifact's `vis-docs/vis-docs.edn` manifest — so extensions' doc pages are lookup-able too). This namespace exposes that corpus to the MODEL through one observation tool so vis can answer questions about ITSELF — features, configuration, how to write an extension — from its real docs instead of guessing. Progressive disclosure: CORE routes vis questions here; page content is only paid for when actually fetched.
Declarative session-level slash commands shared by every channel.
These are channel-agnostic: the engine dispatches them for every channel
through the same slash/dispatch path, and each
handler mutates state via the gateway so the change fans out everywhere.
/rename <new title> set this session's title
/rename routes through titling/set-title-with-broadcast! — the single
title mutation point.
Declarative session-level slash commands shared by every channel. These are channel-agnostic: the engine dispatches them for every channel through the same `slash/dispatch` path, and each handler mutates state via the gateway so the change fans out everywhere. /rename <new title> set this session's title `/rename` routes through `titling/set-title-with-broadcast!` — the single title mutation point.
shell/ compatibility extension — a DROPPABLE classpath plug-in (drop the
jar, drop the feature). Bound only when the user-owned shell toggle is ON
(default ON; flip it OFF in Settings or in vis.yml via toggles: {shell: false}
to drop the tools). The OS process jail is the containment layer while active.
ONE model-facing binding — shell — bound BARE in the flat Python sandbox
next to git / cat / grep. There is no shell_run / shell_bg /
shell_logs / shell_send quartet any more: four names for ONE subsystem
meant four call shapes and four result shapes for what is a single process
lifecycle. One tool, one op grammar, one TOTAL result map:
RUN (default) await shell(cmd) / await shell(cmd, opts) — bash -lc in the
workspace root, waits up to a timeout. Output is bounded at READ time to a
head+tail budget per stream, so only the MIDDLE of a huge stream is
dropped, never its start or end (a chatty-then-killed command cannot
balloon the heap). A non-zero exit is DATA the model reads, not an error.
BACKGROUND await shell(cmd, {"op": "background", "id": "dev"}) — an id
makes it background (the op may stay implicit): spawned under a REAL pty,
its merged output pumped into a bounded ring buffer, registered as a session
RESOURCE in internal.resources (footer count, F4 dialog, resources ctx
block). Prefer this for long builds, test suites, servers, watchers, and
interactive commands; reserve run for short bounded work.
LOGS / SEND / STOP await shell({"op": "logs", "id": "dev"}) — tail the ring
buffer, type into the pty, or kill the tree and drop the resource. Shell stop
and await resource_stop(id) land on the same resources/stop!. An EXITED process
is not auto-pruned, so its output and exit code stay readable until it is
stopped.
Commands have ONE spelling in Python: the first positional argument for
run/background. Resource IDs also have ONE spelling: id inside the options map
for background/logs/send/stop. Native JSON still carries a cmd property; the symbol's
:call shape converts that transport field to the Python positional before
dispatch.
Every op answers with the SAME key set (shell-result-base): keys a stage
does not fill are nil / false / 0 instead of absent, so model Python indexes
any field without a KeyError.
The shell toggle is registered HERE, extension-owned under the vis namespace.
`shell/` compatibility extension — a DROPPABLE classpath plug-in (drop the
jar, drop the feature). Bound only when the user-owned `shell` toggle is ON
(default ON; flip it OFF in Settings or in `vis.yml` via `toggles: {shell: false}`
to drop the tools). The OS process jail is the containment layer while active.
ONE model-facing binding — `shell` — bound BARE in the flat Python sandbox
next to `git` / `cat` / `grep`. There is no `shell_run` / `shell_bg` /
`shell_logs` / `shell_send` quartet any more: four names for ONE subsystem
meant four call shapes and four result shapes for what is a single process
lifecycle. One tool, one `op` grammar, one TOTAL result map:
1. RUN (default) `await shell(cmd)` / `await shell(cmd, opts)` — `bash -lc` in the
workspace root, waits up to a timeout. Output is bounded at READ time to a
head+tail budget per stream, so only the MIDDLE of a huge stream is
dropped, never its start or end (a chatty-then-killed command cannot
balloon the heap). A non-zero exit is DATA the model reads, not an error.
2. BACKGROUND `await shell(cmd, {"op": "background", "id": "dev"})` — an `id`
makes it background (the op may stay implicit): spawned under a REAL pty,
its merged output pumped into a bounded ring buffer, registered as a session
RESOURCE in `internal.resources` (footer count, F4 dialog, `resources` ctx
block). Prefer this for long builds, test suites, servers, watchers, and
interactive commands; reserve run for short bounded work.
3. LOGS / SEND / STOP `await shell({"op": "logs", "id": "dev"})` — tail the ring
buffer, type into the pty, or kill the tree and drop the resource. Shell stop
and `await resource_stop(id)` land on the same `resources/stop!`. An EXITED process
is not auto-pruned, so its output and exit code stay readable until it is
stopped.
Commands have ONE spelling in Python: the first positional argument for
run/background. Resource IDs also have ONE spelling: `id` inside the options map
for background/logs/send/stop. Native JSON still carries a `cmd` property; the symbol's
`:call` shape converts that transport field to the Python positional before
dispatch.
Every op answers with the SAME key set (`shell-result-base`): keys a stage
does not fill are nil / false / 0 instead of absent, so model Python indexes
any field without a KeyError.
The `shell` toggle is registered HERE, extension-owned under the vis namespace.Built-in sandbox SHIM: vis_attach / vis_attach_bytes — the GENERIC
producer twin of the matplotlib capture. A tool running in python_execution
writes any artifact (a PNG it rendered, a CSV/JSON/PDF/wav it built, whatever)
and hands it to vis_attach(path) (or vis_attach_bytes(data, name)); the
engine then OWNS the bytes as a durable session_iteration_attachment row,
exactly like a matplotlib figure — surviving a web/TUI restart and (for image
media-types) replayable to a vision model cross-turn.
No parsing, no round-trip through the model-facing stdout: we control the whole
boundary. The Python side reads the file through the sandbox's OWN confined
open (so filesystem-root confinement is enforced for free — a path outside
the roots raises the normal sandbox error), sniffs the media-type (magic bytes
then extension then utf-8 probe), base64-encodes, and calls the tiny host
bridge __vis_record_attachment__, which appends the attachment map to the
per-block *image-sink* (mpl-capture/record-attachment!). run-python-block
drains that sink into the block outcome's :attachments; the loop stamps each with
the producing block's tool-call-id and hands them to db-store-iteration!'s
:attachments.
Registered unconditionally as a foundation shim (like shim-yaml /
shim-matplotlib): its :ext/sandbox-shims entry autoloads vis_attach into
every sandbox (main + every sub_loop fork).
Built-in sandbox SHIM: `vis_attach` / `vis_attach_bytes` — the GENERIC producer twin of the matplotlib capture. A tool running in `python_execution` writes any artifact (a PNG it rendered, a CSV/JSON/PDF/wav it built, whatever) and hands it to `vis_attach(path)` (or `vis_attach_bytes(data, name)`); the engine then OWNS the bytes as a durable `session_iteration_attachment` row, exactly like a matplotlib figure — surviving a web/TUI restart and (for image media-types) replayable to a vision model cross-turn. No parsing, no round-trip through the model-facing stdout: we control the whole boundary. The Python side reads the file through the sandbox's OWN confined `open` (so filesystem-root confinement is enforced for free — a path outside the roots raises the normal sandbox error), sniffs the media-type (magic bytes then extension then utf-8 probe), base64-encodes, and calls the tiny host bridge `__vis_record_attachment__`, which appends the attachment map to the per-block `*image-sink*` (`mpl-capture/record-attachment!`). `run-python-block` drains that sink into the block outcome's `:attachments`; the loop stamps each with the producing block's tool-call-id and hands them to `db-store-iteration!`'s `:attachments`. Registered unconditionally as a foundation shim (like shim-yaml / shim-matplotlib): its `:ext/sandbox-shims` entry autoloads `vis_attach` into every sandbox (main + every `sub_loop` fork).
Built-in sandbox SHIM: a bs4 (BeautifulSoup)-compatible module for the
model's Python sandbox, implemented in PURE Python on the stdlib
html.parser — NO host/JVM bridge, NOT a line of Clojure or babashka. bs4 is
a third-party wheel that does not ship in GraalPy, so agents that reach for
from bs4 import BeautifulSoup (the natural partner to the requests shim:
fetch then parse) would otherwise hit ModuleNotFoundError; this extension
contributes a :ext/sandbox-shims entry that env-python/build-agent-context
installs into every sandbox Context (main + every sub_loop fork).
It builds a Tag / NavigableString tree via html.parser, with
find/find_all (name/attrs/class_/id/string/recursive/limit), CSS .select
/ .select_one (type / #id / .class / [attr]/[attr=v]/~=/^=/$=/
*=, descendant + > child combinators, comma groups), get_text,
.string/.strings/.stripped_strings, sibling/parent navigation, dynamic
soup.tagname access, and HTML serialization. A deliberate subset of full
bs4 (no lxml, no advanced CSS pseudo-classes).
Like shim-requests there are NO :shim/bindings: the shim is a
self-contained Python preamble with zero host callables. It publishes a bs4
module (+ bs4.element) into sys.modules (so from bs4 import BeautifulSoup
works) and staples BeautifulSoup onto builtins.
Built-in sandbox SHIM: a `bs4` (BeautifulSoup)-compatible module for the model's Python sandbox, implemented in PURE Python on the stdlib `html.parser` — NO host/JVM bridge, NOT a line of Clojure or babashka. bs4 is a third-party wheel that does not ship in GraalPy, so agents that reach for `from bs4 import BeautifulSoup` (the natural partner to the `requests` shim: fetch then parse) would otherwise hit ModuleNotFoundError; this extension contributes a `:ext/sandbox-shims` entry that `env-python/build-agent-context` installs into every sandbox Context (main + every `sub_loop` fork). It builds a `Tag` / `NavigableString` tree via `html.parser`, with `find`/`find_all` (name/attrs/class_/id/string/recursive/limit), CSS `.select` / `.select_one` (type / `#id` / `.class` / `[attr]`/`[attr=v]`/`~=`/`^=`/`$=`/ `*=`, descendant + `>` child combinators, comma groups), `get_text`, `.string`/`.strings`/`.stripped_strings`, sibling/parent navigation, dynamic `soup.tagname` access, and HTML serialization. A deliberate subset of full bs4 (no lxml, no advanced CSS pseudo-classes). Like `shim-requests` there are NO `:shim/bindings`: the shim is a self-contained Python preamble with zero host callables. It publishes a `bs4` module (+ `bs4.element`) into `sys.modules` (so `from bs4 import BeautifulSoup` works) and staples `BeautifulSoup` onto builtins.
Built-in sandbox SHIM: a fontTools/brotli pair for the model's Python
sandbox, scoped to WOFF2 -> TTF conversion. The fonttools and brotli
PyPI packages are not in GraalPy and cannot be pip-installed, so an agent
that reaches for from fontTools.ttLib.woff2 import decompress (or
import brotli) would otherwise hit ModuleNotFoundError. This extension
contributes a :ext/sandbox-shims entry that env-python installs into every
sandbox Context (main + every sub_loop fork).
Why this exists: WOFF2 web fonts are Brotli-compressed sfnt data, so PIL
(ImageFont.truetype) cannot load a .woff2 directly - it needs a real
.ttf/.otf. Converting WOFF2 -> TTF is exactly what fontTools' woff2 module
does, and it needs a Brotli decoder. Both are absent, so brand text rendering
with PIL dead-ends. This shim closes that gap fully in pure Python.
Backing: the vendored brotlidecpy decoder (Sidney Markowitz, MIT) provides
Brotli decompression and its 122784-byte dictionary; an inlined WOFF2 reader
rebuilds the sfnt, undoing the glyf/loca transforms (composite glyphs copied
verbatim, simple glyphs re-encoded). Verified against fontTools 4.63 on
Inter, Roboto (hinted) and NotoSans - ~10800 glyphs, identical outlines; the
reconstructed TTF renders pixel-identical to the vendor TTF under the PIL
shim on GraalPy.
Supported: fontTools.ttLib.woff2.decompress(input, output) and
brotli.decompress(bytes). NOT supported: brotli.compress (raises
NotImplementedError), WOFF1, and the wider fontTools.ttLib.TTFont API -
this is a correctness-focused WOFF2->TTF subset, not full fontTools.
Like shim-pil there are NO :shim/bindings: a self-contained Python
preamble with zero host callables. Publishes fontTools, fontTools.ttLib,
fontTools.ttLib.woff2 and brotli (plus brotlidecpy) into sys.modules
and staples fontTools/brotli onto builtins. No pip, no native wheel, no
host binary.
Built-in sandbox SHIM: a `fontTools`/`brotli` pair for the model's Python sandbox, scoped to WOFF2 -> TTF conversion. The `fonttools` and `brotli` PyPI packages are not in GraalPy and cannot be pip-installed, so an agent that reaches for `from fontTools.ttLib.woff2 import decompress` (or `import brotli`) would otherwise hit ModuleNotFoundError. This extension contributes a `:ext/sandbox-shims` entry that env-python installs into every sandbox Context (main + every `sub_loop` fork). Why this exists: WOFF2 web fonts are Brotli-compressed sfnt data, so PIL (`ImageFont.truetype`) cannot load a `.woff2` directly - it needs a real `.ttf`/`.otf`. Converting WOFF2 -> TTF is exactly what fontTools' woff2 module does, and it needs a Brotli decoder. Both are absent, so brand text rendering with PIL dead-ends. This shim closes that gap fully in pure Python. Backing: the vendored `brotlidecpy` decoder (Sidney Markowitz, MIT) provides Brotli decompression and its 122784-byte dictionary; an inlined WOFF2 reader rebuilds the sfnt, undoing the glyf/loca transforms (composite glyphs copied verbatim, simple glyphs re-encoded). Verified against fontTools 4.63 on Inter, Roboto (hinted) and NotoSans - ~10800 glyphs, identical outlines; the reconstructed TTF renders pixel-identical to the vendor TTF under the PIL shim on GraalPy. Supported: `fontTools.ttLib.woff2.decompress(input, output)` and `brotli.decompress(bytes)`. NOT supported: `brotli.compress` (raises NotImplementedError), WOFF1, and the wider `fontTools.ttLib.TTFont` API - this is a correctness-focused WOFF2->TTF subset, not full fontTools. Like `shim-pil` there are NO `:shim/bindings`: a self-contained Python preamble with zero host callables. Publishes `fontTools`, `fontTools.ttLib`, `fontTools.ttLib.woff2` and `brotli` (plus `brotlidecpy`) into `sys.modules` and staples `fontTools`/`brotli` onto builtins. No pip, no native wheel, no host binary.
Built-in sandbox SHIM: an httpx-compatible module for the model's Python
sandbox, implemented as a thin synchronous wrapper over the already-installed
requests shim (which itself rides the sandbox socket via stdlib urllib and
honours the network guard). No pip, no native wheel, no host bridge.
The preamble publishes an httpx module into sys.modules (so import httpx
and httpx.get(...) work) and staples it onto builtins. It exposes the sync
surface agents actually reach for: module-level get/post/put/patch/delete/ head/options/request, a Client (with base_url, default headers/params,
context-manager support), an httpx-style Response (.status_code, .text,
.content, .json(), .headers, .url, .is_success/.is_error/.is_redirect,
.raise_for_status()), Headers, URL, Timeout, and the httpx exception
tree (HTTPError, RequestError, HTTPStatusError, TimeoutException,
ConnectError). Async is supported too: an AsyncClient whose request/get/ post/put/patch/delete/head/options are awaitable coroutines (with aclose and
async with support) over the same sync core.
Built-in sandbox SHIM: an `httpx`-compatible module for the model's Python sandbox, implemented as a thin synchronous wrapper over the already-installed `requests` shim (which itself rides the sandbox socket via stdlib urllib and honours the network guard). No pip, no native wheel, no host bridge. The preamble publishes an `httpx` module into `sys.modules` (so `import httpx` and `httpx.get(...)` work) and staples it onto builtins. It exposes the sync surface agents actually reach for: module-level `get/post/put/patch/delete/ head/options/request`, a `Client` (with `base_url`, default headers/params, context-manager support), an httpx-style `Response` (`.status_code`, `.text`, `.content`, `.json()`, `.headers`, `.url`, `.is_success/.is_error/.is_redirect`, `.raise_for_status()`), `Headers`, `URL`, `Timeout`, and the `httpx` exception tree (`HTTPError`, `RequestError`, `HTTPStatusError`, `TimeoutException`, `ConnectError`). Async is supported too: an `AsyncClient` whose `request/get/ post/put/patch/delete/head/options` are awaitable coroutines (with `aclose` and `async with` support) over the same sync core.
Built-in sandbox SHIM: a minimal matplotlib.pyplot-compatible module for the
model's Python sandbox, backed by a pure-JVM Java2D renderer. The agent
sandbox ships no CPython matplotlib wheel (it needs numpy's native core +
freetype, all blocked by the deny-by-default Context); this extension instead
contributes a :ext/sandbox-shims entry whose Python preamble accumulates the
familiar pyplot artists (plot/scatter/bar/hist/fill_between/step/
pie/axhline/axvline/title/xlabel/… plus the OO subplots/Axes
API) and whose savefig DELEGATES the whole figure spec across the boundary
to the host callable __vis_mpl_render__, which draws it with java.awt/
ImageIO and returns a base64 PNG. The Python side base64-decodes and writes
it (to a path, confined to the sandbox roots, or to any file-like buffer).
It is a SUBSET, not real matplotlib: line/scatter/bar/hist/fill/step/pie with title, axis labels, grid, legend, dashed line styles, markers, log scales and text annotations — enough for the model to visualize data. Rendering runs entirely on the JVM (no pip, no native wheels), and any render failure surfaces to Python as a catchable exception (never crashes the sandbox).
Together with shim-yaml this demonstrates the sandbox-shim mechanism: an
extension turns a host / JVM capability into a real importable Python module
while env-python stays completely generic about which shims exist.
Built-in sandbox SHIM: a minimal `matplotlib.pyplot`-compatible module for the model's Python sandbox, backed by a pure-JVM Java2D renderer. The agent sandbox ships no CPython matplotlib wheel (it needs numpy's native core + freetype, all blocked by the deny-by-default Context); this extension instead contributes a `:ext/sandbox-shims` entry whose Python preamble accumulates the familiar pyplot artists (`plot`/`scatter`/`bar`/`hist`/`fill_between`/`step`/ `pie`/`axhline`/`axvline`/`title`/`xlabel`/… plus the OO `subplots`/`Axes` API) and whose `savefig` DELEGATES the whole figure spec across the boundary to the host callable `__vis_mpl_render__`, which draws it with `java.awt`/ `ImageIO` and returns a base64 PNG. The Python side base64-decodes and writes it (to a path, confined to the sandbox roots, or to any file-like buffer). It is a SUBSET, not real matplotlib: line/scatter/bar/hist/fill/step/pie with title, axis labels, grid, legend, dashed line styles, markers, log scales and text annotations — enough for the model to visualize data. Rendering runs entirely on the JVM (no pip, no native wheels), and any render failure surfaces to Python as a catchable exception (never crashes the sandbox). Together with `shim-yaml` this demonstrates the sandbox-shim mechanism: an extension turns a host / JVM capability into a real importable Python module while `env-python` stays completely generic about which shims exist.
Built-in sandbox SHIM exposing Vis's Nippy persistence codec to Python.
nippy_decode(bytes) decodes trusted Vis-owned Nippy BLOBs (for example
session_turn_iteration.forms, session_turn_state.ctx, and
session_turn_state.error) into native Python data. Nippy-stream Vectorz
vectors decode as Python lists. nippy_encode(value) performs the inverse for
Python plain data. The same functions are available as nippy.decode /
nippy.encode after import nippy.
The Python module and Vectorz codec registration are lazy: neither runs during sandbox context initialization; Vectorz installs on the first codec call. Decoded Clojure values cross the normal sandbox boundary: map keys become canonical snake_case strings, keyword/symbol values become strings, dates become epoch milliseconds, and unsupported leaves stringify. This is for inspection and plain-data round trips, not exact Clojure type preservation. Java Serializable fallback is disabled in both directions.
Built-in sandbox SHIM exposing Vis's Nippy persistence codec to Python. `nippy_decode(bytes)` decodes trusted Vis-owned Nippy BLOBs (for example `session_turn_iteration.forms`, `session_turn_state.ctx`, and `session_turn_state.error`) into native Python data. Nippy-stream Vectorz vectors decode as Python lists. `nippy_encode(value)` performs the inverse for Python plain data. The same functions are available as `nippy.decode` / `nippy.encode` after `import nippy`. The Python module and Vectorz codec registration are lazy: neither runs during sandbox context initialization; Vectorz installs on the first codec call. Decoded Clojure values cross the normal sandbox boundary: map keys become canonical snake_case strings, keyword/symbol values become strings, dates become epoch milliseconds, and unsupported leaves stringify. This is for inspection and plain-data round trips, not exact Clojure type preservation. Java Serializable fallback is disabled in both directions.
Built-in sandbox SHIM: a numpy-compatible module for the model's Python
sandbox, implemented in PURE Python (stdlib math + random) — NO host/JVM
bridge, NOT a line of Clojure or babashka. numpy is a native C wheel that does
not ship in GraalPy, so agents that reach for import numpy would otherwise
hit ModuleNotFoundError; this extension contributes a :ext/sandbox-shims
entry that env-python/build-agent-context installs into every sandbox
Context (main + every sub_loop fork).
The shim is a correctness-focused SUBSET, not a C-speed numpy: an ndarray
backed by a flat Python list + shape tuple, with broadcasting, reductions,
ufuncs, fancy/boolean/slice indexing, dot/matmul, a linalg submodule
(norm/det/inv/solve/matrix_power/matrix_rank via pure-Python Gaussian
elimination) and a random submodule (stdlib random). Big arrays are slow;
the goal is that agent glue code (np.array, arithmetic, mean/sum, small
linear algebra) just works.
Like shim-requests there are NO :shim/bindings: the shim is a
self-contained Python preamble with zero host callables. It publishes a
numpy module into sys.modules (so import numpy works) and staples it
onto builtins (so numpy.array(...) works with NO import, like json/os).
Built-in sandbox SHIM: a `numpy`-compatible module for the model's Python sandbox, implemented in PURE Python (stdlib math + random) — NO host/JVM bridge, NOT a line of Clojure or babashka. numpy is a native C wheel that does not ship in GraalPy, so agents that reach for `import numpy` would otherwise hit ModuleNotFoundError; this extension contributes a `:ext/sandbox-shims` entry that `env-python/build-agent-context` installs into every sandbox Context (main + every `sub_loop` fork). The shim is a correctness-focused SUBSET, not a C-speed numpy: an `ndarray` backed by a flat Python list + shape tuple, with broadcasting, reductions, ufuncs, fancy/boolean/slice indexing, `dot`/`matmul`, a `linalg` submodule (norm/det/inv/solve/matrix_power/matrix_rank via pure-Python Gaussian elimination) and a `random` submodule (stdlib random). Big arrays are slow; the goal is that agent glue code (`np.array`, arithmetic, `mean`/`sum`, small linear algebra) just works. Like `shim-requests` there are NO `:shim/bindings`: the shim is a self-contained Python preamble with zero host callables. It publishes a `numpy` module into `sys.modules` (so `import numpy` works) and staples it onto builtins (so `numpy.array(...)` works with NO import, like json/os).
Built-in sandbox SHIM: a pandas-compatible module for the model's Python
sandbox, implemented in PURE Python (stdlib csv/json/math) — NO host/JVM
bridge, NOT a line of Clojure or babashka. pandas is a native/heavy wheel that
does not ship in GraalPy, so agents that reach for import pandas would
otherwise hit ModuleNotFoundError; this extension contributes a
:ext/sandbox-shims entry that env-python installs into every sandbox Context
(main + every sub_loop fork).
The shim is a correctness-focused SUBSET, not C-speed pandas: a Series is a
labelled 1-D column, a DataFrame is an ordered dict of columns. It covers
construction (dict / records / list-of-lists / read_csv / read_json),
[]/loc/iloc selection, boolean masking, column arithmetic, groupby
(sum/mean/min/max/count/size/agg), merge (inner/left/right/outer),
concat, sort_values, describe, fillna/dropna, apply, a .str
accessor, to_dict/to_csv/to_json and a pandas-style __repr__. It
interoperates with the numpy shim (.values) when present. Big frames are
slow; the goal is that agent glue code just works.
Like shim-numpy there are NO :shim/bindings: the shim is a self-contained
Python preamble with zero host callables. It publishes a pandas module into
sys.modules (so import pandas works) and staples it onto builtins (so
pandas.DataFrame(...) works with NO import, like json/os).
Built-in sandbox SHIM: a `pandas`-compatible module for the model's Python sandbox, implemented in PURE Python (stdlib csv/json/math) — NO host/JVM bridge, NOT a line of Clojure or babashka. pandas is a native/heavy wheel that does not ship in GraalPy, so agents that reach for `import pandas` would otherwise hit ModuleNotFoundError; this extension contributes a `:ext/sandbox-shims` entry that env-python installs into every sandbox Context (main + every `sub_loop` fork). The shim is a correctness-focused SUBSET, not C-speed pandas: a `Series` is a labelled 1-D column, a `DataFrame` is an ordered dict of columns. It covers construction (dict / records / list-of-lists / read_csv / read_json), `[]`/`loc`/`iloc` selection, boolean masking, column arithmetic, `groupby` (sum/mean/min/max/count/size/agg), `merge` (inner/left/right/outer), `concat`, `sort_values`, `describe`, `fillna`/`dropna`, `apply`, a `.str` accessor, `to_dict`/`to_csv`/`to_json` and a pandas-style `__repr__`. It interoperates with the numpy shim (`.values`) when present. Big frames are slow; the goal is that agent glue code just works. Like `shim-numpy` there are NO `:shim/bindings`: the shim is a self-contained Python preamble with zero host callables. It publishes a `pandas` module into `sys.modules` (so `import pandas` works) and staples it onto builtins (so `pandas.DataFrame(...)` works with NO import, like json/os).
Built-in sandbox SHIM: a paramiko-compatible SSH2 module backed by the
pure-Java mwiede JSch fork (com.github.mwiede/jsch) so import paramiko
works without the native CPython cryptography/cffi wheels. SSH sessions and
SFTP channels live HOST-side (JSch Session/ChannelSftp in integer-keyed
registries); the Python classes are thin handle wrappers, exchanging
command/path strings and base64 file bytes across the boundary.
Built-in sandbox SHIM: a `paramiko`-compatible SSH2 module backed by the pure-Java mwiede JSch fork (`com.github.mwiede/jsch`) so `import paramiko` works without the native CPython cryptography/cffi wheels. SSH sessions and SFTP channels live HOST-side (JSch `Session`/`ChannelSftp` in integer-keyed registries); the Python classes are thin handle wrappers, exchanging command/path strings and base64 file bytes across the boundary.
Built-in sandbox SHIM: a Pillow (PIL)-compatible PIL package for the model's
Python sandbox, backed by the JVM's Java2D / ImageIO image stack. No CPython
Pillow wheel ships in the sandbox; this extension contributes a
:ext/sandbox-shims entry that env-python/build-agent-context installs into
every sandbox Context (main + every sub_loop fork): the host bridge callables
are wired onto the globals, then the Python preamble publishes a PIL package
(with Image, ImageDraw, ImageFilter, ImageOps, ImageColor,
ImageEnhance, ImageChops, ImageFont, ImageMath submodules) into
sys.modules (so from PIL import Image works) and staples them onto builtins.
Images live HOST-side as BufferedImages in a per-JVM registry keyed by an
integer handle; the Python Image object is a thin handle wrapper. All pixel
ops, drawing, filtering, geometry and codec work happen on the JVM; only small
metadata vectors and base64 blobs cross the strings-only boundary. Mirrors the
shim-matplotlib Java2D approach and reuses mpl-capture/record-attachment!
so Image.show() surfaces the image inline as a session attachment.
Built-in sandbox SHIM: a Pillow (PIL)-compatible `PIL` package for the model's Python sandbox, backed by the JVM's Java2D / ImageIO image stack. No CPython Pillow wheel ships in the sandbox; this extension contributes a `:ext/sandbox-shims` entry that `env-python/build-agent-context` installs into every sandbox Context (main + every `sub_loop` fork): the host bridge callables are wired onto the globals, then the Python preamble publishes a `PIL` package (with `Image`, `ImageDraw`, `ImageFilter`, `ImageOps`, `ImageColor`, `ImageEnhance`, `ImageChops`, `ImageFont`, `ImageMath` submodules) into `sys.modules` (so `from PIL import Image` works) and staples them onto builtins. Images live HOST-side as `BufferedImage`s in a per-JVM registry keyed by an integer handle; the Python `Image` object is a thin handle wrapper. All pixel ops, drawing, filtering, geometry and codec work happen on the JVM; only small metadata vectors and base64 blobs cross the strings-only boundary. Mirrors the `shim-matplotlib` Java2D approach and reuses `mpl-capture/record-attachment!` so `Image.show()` surfaces the image inline as a session attachment.
Built-in sandbox SHIM: a pptx (python-pptx) compatible module backed by
Apache POI XSLF (org.apache.poi/poi-ooxml) so from pptx import Presentation
writes real .pptx files without the CPython package. Presentations/slides/
shapes live HOST-side in an integer registry; the Python classes are thin
handle wrappers exchanging EMU geometry + base64 image/file bytes across the
boundary.
Built-in sandbox SHIM: a `pptx` (python-pptx) compatible module backed by Apache POI XSLF (`org.apache.poi/poi-ooxml`) so `from pptx import Presentation` writes real .pptx files without the CPython package. Presentations/slides/ shapes live HOST-side in an integer registry; the Python classes are thin handle wrappers exchanging EMU geometry + base64 image/file bytes across the boundary.
Built-in sandbox SHIM: a pytest-compatible module for the model's Python
sandbox, implemented PURELY in Python on the stdlib (ast, inspect,
linecache, traceback, warnings, tempfile) — NO host/JVM bridge, NOT a
line of Clojure or babashka. pytest is a third-party wheel that does not
ship in GraalPy, so agents writing full Python extensions and wanting to
TEST them inline would otherwise hit ModuleNotFoundError; this extension
contributes a :ext/sandbox-shims entry that
env-python/build-agent-context installs into every sandbox Context (main +
every sub_loop fork).
It is NOT real pytest: there is no pluggy/plugin system, no ini/plugin CLI
(only -k / -x / --maxfail / -v), and no import-time assertion
rewrite. It DOES do conftest.py fixture discovery
(walked from the test file's dir up to the fs root, outer→inner) in disk
mode. Instead it
implements the subset that matters in an inline sandbox where the model
writes tests + pytest.main() in ONE block:
test_* functions and Test* classes (scoped to the
CURRENT block via __vis_src__, so leftovers from earlier blocks in the
shared globals are NOT swept in),assert 2 == 3 reconstructed with operand
values) done by registering __vis_src__ into linecache and walking
the failing frame's AST — the same UX as pytest's rewrite, via a
different mechanism,pytest.raises / warns / approx / fail / skip / xfail /
importorskip, @pytest.fixture (function/module/session scope,
yield-teardown, autouse, recursive injection, parametrized fixtures via
params/ids with request.param, request.getfixturevalue chains),
@pytest.mark.parametrize (incl. indirect=) / skip / skipif / xfail
/ usefixtures (+ arbitrary marks), pytest.param,
builtin fixtures request / monkeypatch / capsys / capfd /
tmp_path / tmp_path_factory / tmpdir / tmpdir_factory /
caplog / recwarn / pytester / testdir, conftest.py fixture discovery,
and a pytest.main() runner (with -k keyword selection, -x /
--maxfail fail-fast, and -v) that prints progress + failure reports
Unlike shim-yaml/shim-matplotlib there are NO :shim/bindings: the shim
is a self-contained Python preamble with zero host callables. It publishes a
pytest module into sys.modules (so import pytest works) and staples it
onto builtins (so pytest.raises(...) works with NO import, like
json/os/requests).
Built-in sandbox SHIM: a `pytest`-compatible module for the model's Python
sandbox, implemented PURELY in Python on the stdlib (`ast`, `inspect`,
`linecache`, `traceback`, `warnings`, `tempfile`) — NO host/JVM bridge, NOT a
line of Clojure or babashka. `pytest` is a third-party wheel that does not
ship in GraalPy, so agents writing full Python extensions and wanting to
TEST them inline would otherwise hit ModuleNotFoundError; this extension
contributes a `:ext/sandbox-shims` entry that
`env-python/build-agent-context` installs into every sandbox Context (main +
every `sub_loop` fork).
It is NOT real pytest: there is no pluggy/plugin system, no ini/plugin CLI
(only `-k` / `-x` / `--maxfail` / `-v`), and no import-time assertion
rewrite. It DOES do `conftest.py` fixture discovery
(walked from the test file's dir up to the fs root, outer→inner) in disk
mode. Instead it
implements the subset that matters in an inline sandbox where the model
writes tests + `pytest.main()` in ONE block:
- collection of `test_*` functions and `Test*` classes (scoped to the
CURRENT block via `__vis_src__`, so leftovers from earlier blocks in the
shared globals are NOT swept in),
- RUNTIME assert introspection (`assert 2 == 3` reconstructed with operand
values) done by registering `__vis_src__` into `linecache` and walking
the failing frame's AST — the same UX as pytest's rewrite, via a
different mechanism,
- `pytest.raises` / `warns` / `approx` / `fail` / `skip` / `xfail` /
`importorskip`, `@pytest.fixture` (function/module/session scope,
yield-teardown, autouse, recursive injection, parametrized fixtures via
`params`/`ids` with `request.param`, `request.getfixturevalue` chains),
`@pytest.mark.parametrize` (incl. `indirect=`) / `skip` / `skipif` / `xfail`
/ `usefixtures` (+ arbitrary marks), `pytest.param`,
builtin fixtures `request` / `monkeypatch` / `capsys` / `capfd` /
`tmp_path` / `tmp_path_factory` / `tmpdir` / `tmpdir_factory` /
`caplog` / `recwarn` / `pytester` / `testdir`, `conftest.py` fixture discovery,
and a `pytest.main()` runner (with `-k` keyword selection, `-x` /
`--maxfail` fail-fast, and `-v`) that prints progress + failure reports
+ a summary line (incl. deselected counts) and returns an exit code.
Unlike `shim-yaml`/`shim-matplotlib` there are NO `:shim/bindings`: the shim
is a self-contained Python preamble with zero host callables. It publishes a
`pytest` module into `sys.modules` (so `import pytest` works) and staples it
onto builtins (so `pytest.raises(...)` works with NO import, like
json/os/requests).Built-in sandbox SHIM: a requests-compatible module for the model's Python
sandbox, backed PURELY by the stdlib urllib.request — NO host/JVM bridge,
NOT a line of Clojure or babashka. requests is a third-party wheel that does
not ship in GraalPy, so agents that reach for import requests out of habit
would otherwise hit ModuleNotFoundError; this extension contributes a
:ext/sandbox-shims entry that env-python/build-agent-context installs into
every sandbox Context (main + every sub_loop fork).
Because every call travels through the sandbox's OWN socket (urllib ->
http.client -> socket), it automatically honours the network toggle
(allowHostSocketAccess) AND the allow/deny + anti-SSRF network-guard-python
— a JVM babashka.http-client bridge would open an egress path OUTSIDE the
sandbox and disarm all of that, which is exactly why this stays 100% Python.
Unlike shim-yaml/shim-matplotlib there are NO :shim/bindings: the shim is
a self-contained Python preamble with zero host callables. It publishes a
requests module into sys.modules (so import requests works) and staples
it onto builtins (so requests.get(...) works with NO import, like json/os).
Built-in sandbox SHIM: a `requests`-compatible module for the model's Python sandbox, backed PURELY by the stdlib `urllib.request` — NO host/JVM bridge, NOT a line of Clojure or babashka. `requests` is a third-party wheel that does not ship in GraalPy, so agents that reach for `import requests` out of habit would otherwise hit ModuleNotFoundError; this extension contributes a `:ext/sandbox-shims` entry that `env-python/build-agent-context` installs into every sandbox Context (main + every `sub_loop` fork). Because every call travels through the sandbox's OWN socket (urllib -> http.client -> socket), it automatically honours the network toggle (`allowHostSocketAccess`) AND the allow/deny + anti-SSRF `network-guard-python` — a JVM `babashka.http-client` bridge would open an egress path OUTSIDE the sandbox and disarm all of that, which is exactly why this stays 100% Python. Unlike `shim-yaml`/`shim-matplotlib` there are NO `:shim/bindings`: the shim is a self-contained Python preamble with zero host callables. It publishes a `requests` module into `sys.modules` (so `import requests` works) and staples it onto builtins (so `requests.get(...)` works with NO import, like json/os).
Built-in sandbox SHIM: a DB-API 2.0 sqlite3 module for the model's Python
sandbox, backed by the JVM's xerial sqlite-jdbc driver (already on the
classpath via the persistence extension, so no new dependency and native-image
reachability is already configured). CPython's _sqlite3 native extension is
absent in GraalPy, so import sqlite3 otherwise fails with ModuleNotFoundError.
Connections live HOST-side as java.sql.Connections in a per-JVM registry keyed
by an integer handle; the Python Connection/Cursor are thin handle wrappers.
SQL + params cross the strings-only boundary; result rows come back as vectors
(BLOBs base64-tagged). :memory: databases are fully supported; a file path is
opened host-side via jdbc:sqlite:<path>. Autocommit is on, so commit() is a
no-op flush and data persists without it (the forgiving DB-API path).
Built-in sandbox SHIM: a DB-API 2.0 `sqlite3` module for the model's Python sandbox, backed by the JVM's xerial `sqlite-jdbc` driver (already on the classpath via the persistence extension, so no new dependency and native-image reachability is already configured). CPython's `_sqlite3` native extension is absent in GraalPy, so `import sqlite3` otherwise fails with ModuleNotFoundError. Connections live HOST-side as `java.sql.Connection`s in a per-JVM registry keyed by an integer handle; the Python `Connection`/`Cursor` are thin handle wrappers. SQL + params cross the strings-only boundary; result rows come back as vectors (BLOBs base64-tagged). `:memory:` databases are fully supported; a file path is opened host-side via `jdbc:sqlite:<path>`. Autocommit is on, so `commit()` is a no-op flush and data persists without it (the forgiving DB-API path).
Built-in sandbox SHIM: a tabulate-compatible module for the model's Python
sandbox, implemented in PURE Python (stdlib only) — NO host/JVM bridge. The
tabulate PyPI package is not in GraalPy, so agents that reach for
from tabulate import tabulate would otherwise hit ModuleNotFoundError; this
extension contributes a :ext/sandbox-shims entry that env-python installs
into every sandbox Context (main + every sub_loop fork).
The shim covers the tablefmts agents reach for most: plain, simple, github,
pipe, orgtbl, presto, grid, fancy_grid, rst, tsv and html, with numeric /
string alignment, floatfmt, showindex, and headers='keys'/'firstrow'.
It accepts list-of-lists, list-of-dicts, dict-of-lists, and duck-types the
pandas shim's DataFrame. A correctness-focused SUBSET, not full tabulate.
Like shim-numpy there are NO :shim/bindings: a self-contained Python
preamble with zero host callables. Publishes a tabulate module into
sys.modules (so import tabulate works) and staples the tabulate fn onto
builtins.
Built-in sandbox SHIM: a `tabulate`-compatible module for the model's Python sandbox, implemented in PURE Python (stdlib only) — NO host/JVM bridge. The `tabulate` PyPI package is not in GraalPy, so agents that reach for `from tabulate import tabulate` would otherwise hit ModuleNotFoundError; this extension contributes a `:ext/sandbox-shims` entry that env-python installs into every sandbox Context (main + every `sub_loop` fork). The shim covers the tablefmts agents reach for most: plain, simple, github, pipe, orgtbl, presto, grid, fancy_grid, rst, tsv and html, with numeric / string alignment, `floatfmt`, `showindex`, and `headers='keys'/'firstrow'`. It accepts list-of-lists, list-of-dicts, dict-of-lists, and duck-types the pandas shim's DataFrame. A correctness-focused SUBSET, not full tabulate. Like `shim-numpy` there are NO `:shim/bindings`: a self-contained Python preamble with zero host callables. Publishes a `tabulate` module into `sys.modules` (so `import tabulate` works) and staples the `tabulate` fn onto builtins.
Built-in sandbox SHIM: a toml-compatible module for the model's Python
sandbox — NO host/JVM bridge. The toml PyPI package is not in GraalPy, so
agents that reach for import toml would otherwise hit ModuleNotFoundError;
this extension contributes a :ext/sandbox-shims entry that env-python
installs into every sandbox Context (main + every sub_loop fork).
Reading (toml.loads/toml.load) delegates to the stdlib tomllib (present
in GraalPy's 3.11 stdlib) for a spec-correct parse; writing
(toml.dumps/toml.dump) is a pure-Python serializer covering scalars,
arrays, inline tables, nested [table] sections and [[array.of.tables]].
A correctness-focused SUBSET of the toml package API.
Like shim-numpy there are NO :shim/bindings: a self-contained Python
preamble with zero host callables. Publishes a toml module into
sys.modules (so import toml works) and staples it onto builtins.
Built-in sandbox SHIM: a `toml`-compatible module for the model's Python sandbox — NO host/JVM bridge. The `toml` PyPI package is not in GraalPy, so agents that reach for `import toml` would otherwise hit ModuleNotFoundError; this extension contributes a `:ext/sandbox-shims` entry that env-python installs into every sandbox Context (main + every `sub_loop` fork). Reading (`toml.loads`/`toml.load`) delegates to the stdlib `tomllib` (present in GraalPy's 3.11 stdlib) for a spec-correct parse; writing (`toml.dumps`/`toml.dump`) is a pure-Python serializer covering scalars, arrays, inline tables, nested `[table]` sections and `[[array.of.tables]]`. A correctness-focused SUBSET of the `toml` package API. Like `shim-numpy` there are NO `:shim/bindings`: a self-contained Python preamble with zero host callables. Publishes a `toml` module into `sys.modules` (so `import toml` works) and staples it onto builtins.
Built-in sandbox SHIM: zoneinfo, pytz and dateutil for the model's Python
sandbox, backed by the JVM's java.time IANA time-zone database.
GraalPy ships no writable filesystem, so the real zoneinfo / pytz /
tzdata crash at import (_tzpath calls getcwd, which the denied FS refuses
with an UN-catchable Java SecurityException that aborts the whole eval). This
extension contributes a :ext/sandbox-shims entry that
env-python/build-agent-context installs into every sandbox Context (main +
every sub_loop fork): host callables resolve zone offsets / DST / names via
java.time.ZoneId (604+ zones, no data files), then a Python preamble publishes
zoneinfo, pytz, tzdata and the dateutil package (dateutil.tz,
dateutil.parser, dateutil.relativedelta) into sys.modules and staples them
onto builtins. All tz math happens on the JVM; only small metadata vectors cross
the strings-only boundary. Kills the whole class of tz-aware-datetime failures.
Built-in sandbox SHIM: `zoneinfo`, `pytz` and `dateutil` for the model's Python sandbox, backed by the JVM's `java.time` IANA time-zone database. GraalPy ships no writable filesystem, so the real `zoneinfo` / `pytz` / `tzdata` crash at import (`_tzpath` calls `getcwd`, which the denied FS refuses with an UN-catchable Java `SecurityException` that aborts the whole eval). This extension contributes a `:ext/sandbox-shims` entry that `env-python/build-agent-context` installs into every sandbox Context (main + every `sub_loop` fork): host callables resolve zone offsets / DST / names via `java.time.ZoneId` (604+ zones, no data files), then a Python preamble publishes `zoneinfo`, `pytz`, `tzdata` and the `dateutil` package (`dateutil.tz`, `dateutil.parser`, `dateutil.relativedelta`) into `sys.modules` and staples them onto builtins. All tz math happens on the JVM; only small metadata vectors cross the strings-only boundary. Kills the whole class of tz-aware-`datetime` failures.
Built-in sandbox SHIM: a urllib3-compatible module for the model's Python
sandbox, implemented as a thin wrapper over the already-installed requests
shim (which rides the sandbox socket via stdlib urllib and honours the network
guard). No pip, no native wheel, no host bridge.
The preamble publishes a urllib3 module (plus urllib3.exceptions) into
sys.modules and staples it onto builtins. It exposes the surface agents
reach for: PoolManager / HTTPConnectionPool / HTTPSConnectionPool with
.request(method, url, fields=, headers=, body=, json=), a top-level
urllib3.request(...) (urllib3 2.x), an HTTPResponse (.status, .data,
.headers, .json(), .read(), .getheader()), HTTPHeaderDict,
disable_warnings(), and the urllib3.exceptions tree (HTTPError,
MaxRetryError, NewConnectionError, ReadTimeoutError, ProtocolError,
InsecureRequestWarning). Real connection pooling / retries are no-ops.
Built-in sandbox SHIM: a `urllib3`-compatible module for the model's Python sandbox, implemented as a thin wrapper over the already-installed `requests` shim (which rides the sandbox socket via stdlib urllib and honours the network guard). No pip, no native wheel, no host bridge. The preamble publishes a `urllib3` module (plus `urllib3.exceptions`) into `sys.modules` and staples it onto builtins. It exposes the surface agents reach for: `PoolManager` / `HTTPConnectionPool` / `HTTPSConnectionPool` with `.request(method, url, fields=, headers=, body=, json=)`, a top-level `urllib3.request(...)` (urllib3 2.x), an `HTTPResponse` (`.status`, `.data`, `.headers`, `.json()`, `.read()`, `.getheader()`), `HTTPHeaderDict`, `disable_warnings()`, and the `urllib3.exceptions` tree (`HTTPError`, `MaxRetryError`, `NewConnectionError`, `ReadTimeoutError`, `ProtocolError`, `InsecureRequestWarning`). Real connection pooling / retries are no-ops.
Built-in sandbox SHIM: an xlsxwriter-compatible module backed by Apache POI
(org.apache.poi/poi-ooxml) so import xlsxwriter writes real .xlsx files
without the CPython package. Workbooks/formats live HOST-side in an integer
registry; the Python classes are thin handle wrappers; the finished file
crosses the boundary as base64 bytes on close().
Built-in sandbox SHIM: an `xlsxwriter`-compatible module backed by Apache POI (`org.apache.poi/poi-ooxml`) so `import xlsxwriter` writes real .xlsx files without the CPython package. Workbooks/formats live HOST-side in an integer registry; the Python classes are thin handle wrappers; the finished file crosses the boundary as base64 bytes on `close()`.
Built-in sandbox SHIM: a PyYAML-compatible yaml module for the model's
Python sandbox, backed by the pure-Clojure YAMLStar loader
(org.yamlstar/yamlstar). No CPython PyYAML wheel ships in the sandbox; this
extension contributes a :ext/sandbox-shims entry that
env-python/build-agent-context installs into every sandbox Context (main +
every sub_loop fork): the host bridge callables are wired onto the globals,
then the Python preamble publishes a yaml module into sys.modules (so
import yaml works) and staples it onto builtins (so yaml.safe_load(...)
works with NO import).
This is the reference example of the sandbox-shim mechanism: a host / JVM capability surfaced to sandbox Python as a real importable module, with the engine staying completely generic about which shims exist.
Built-in sandbox SHIM: a PyYAML-compatible `yaml` module for the model's Python sandbox, backed by the pure-Clojure YAMLStar loader (`org.yamlstar/yamlstar`). No CPython PyYAML wheel ships in the sandbox; this extension contributes a `:ext/sandbox-shims` entry that `env-python/build-agent-context` installs into every sandbox Context (main + every `sub_loop` fork): the host bridge callables are wired onto the globals, then the Python preamble publishes a `yaml` module into `sys.modules` (so `import yaml` works) and staples it onto builtins (so `yaml.safe_load(...)` works with NO import). This is the reference example of the sandbox-shim mechanism: a host / JVM capability surfaced to sandbox Python as a real importable module, with the engine staying completely generic about which shims exist.
clojure.spec CONTRACT for the language-surface tool RESULTS (format_code,
lint_code, run_tests).
Every language pack that registers a :format-fn / :lint-fn / :test-fn
under
:ext/language-tools returns a result map that MUST conform to these specs,
so the shape is UNIFORM across packs (clojure, and a future python / js) and
can never silently drift. Both results share the directory-nested by-dir
grouping ({<dir> {<basename> <payload>}}) that writes each long directory
prefix ONCE.
The result maps cross the strings-only Python boundary, so their keys are
STRINGS ("op", "findings", "by-dir", ...). clojure.spec's s/keys only
speaks keyword keys, so the map specs here are plain predicates over the
string keys, composed from s/map-of / s/coll-of for the nested pieces.
check validates a result and returns it UNCHANGED, throwing ex-info with
s/explain-data when it violates the contract — the schema check the packs
run on every format/lint result before handing it back through the surface.
capability->spec is the single source of truth mapping a capability keyword
to its result spec.
clojure.spec CONTRACT for the language-surface tool RESULTS (`format_code`,
`lint_code`, `run_tests`).
Every language pack that registers a `:format-fn` / `:lint-fn` / `:test-fn`
under
`:ext/language-tools` returns a result map that MUST conform to these specs,
so the shape is UNIFORM across packs (clojure, and a future python / js) and
can never silently drift. Both results share the directory-nested `by-dir`
grouping (`{<dir> {<basename> <payload>}}`) that writes each long directory
prefix ONCE.
The result maps cross the strings-only Python boundary, so their keys are
STRINGS ("op", "findings", "by-dir", ...). clojure.spec's `s/keys` only
speaks keyword keys, so the map specs here are plain predicates over the
string keys, composed from `s/map-of` / `s/coll-of` for the nested pieces.
`check` validates a result and returns it UNCHANGED, throwing ex-info with
`s/explain-data` when it violates the contract — the schema check the packs
run on every format/lint result before handing it back through the surface.
`capability->spec` is the single source of truth mapping a capability keyword
to its result spec.Full session transcript - DATA first, presentation second.
transcript returns one canonical Clojure map with every turn,
every iteration, every executed block plus the LLM-side context
(system prompt, message envelope, reasoning trace, top-level
provider error, per-iteration vars, answer-form pointer,
returned-empty-blocks flag) and the per-block forensic detail
(code, comment, result, error, duration, timeout?, repaired?).
Pure data. The agent can pattern-match on it; the CLI
renders Markdown on top; a future TUI screen, JSON exporter, or
analytics extension consumes the same shape.
Lives in foundation because it's an introspection surface, not host
plumbing. The sandbox-visible public surface is (session-state)
for data and (session-report-html) for an HTML report; this namespace
owns the transcript portion behind that deeper interface.
Public Clojure surface:
(transcript db-info session-id) -> transcript data map
(transcript->md data) -> Markdown string
(transcript-md db-info session-id) -> DB lookup + Markdown string
Canonical data shape:
{:session {:id :title :channel :model :provider :created-at} :totals {:turns N :iterations N :tokens {:input :output :reasoning :cached} :cost-usd D} :dialog [{:role :turn-id :content}] :calls [{:kind :ref :parent-ref :turn-id :iteration-id :op :tool :var :code :status :duration-ms :command :target :result :result-summary :info}] :timeline [{:kind :ref :turn-id :iteration-id :content :code :status :duration-ms :result-summary}] :turns [{:id :user-request :status :prior-outcome :provider :model :iteration-count :failure-count :tokens :cost-usd :content :iterations [{:id :position :status :duration-ms :provider :model :thinking :error :tokens :cost-usd :answer-position :returned-empty-blocks? :vars [{:name :code :value :version}] :attachments [{:id :source :tool-call-id :position :kind :media-type :filename :size :stored}] :blocks [{:position :code :comment :result :stdout :error :duration-ms :timeout? :repaired?}]}]}]}
The Markdown renderer renders thinking, iteration-level errors, vars, per-block forensic previews, and final answer text. Large fields are bounded so reports stay safe to open.
Full session transcript - DATA first, presentation second.
`transcript` returns one canonical Clojure map with every turn,
every iteration, every executed block plus the LLM-side context
(system prompt, message envelope, reasoning trace, top-level
provider error, per-iteration vars, answer-form pointer,
returned-empty-blocks flag) and the per-block forensic detail
(code, comment, result, error, duration, timeout?, repaired?).
Pure data. The agent can pattern-match on it; the CLI
renders Markdown on top; a future TUI screen, JSON exporter, or
analytics extension consumes the same shape.
Lives in foundation because it's an introspection surface, not host
plumbing. The sandbox-visible public surface is `(session-state)`
for data and `(session-report-html)` for an HTML report; this namespace
owns the transcript portion behind that deeper interface.
Public Clojure surface:
`(transcript db-info session-id)` -> transcript data map
`(transcript->md data)` -> Markdown string
`(transcript-md db-info session-id)` -> DB lookup + Markdown string
Canonical data shape:
{:session {:id :title :channel :model :provider :created-at}
:totals {:turns N :iterations N
:tokens {:input :output :reasoning :cached}
:cost-usd D}
:dialog [{:role :turn-id :content}]
:calls [{:kind :ref :parent-ref :turn-id :iteration-id :op :tool
:var :code :status :duration-ms :command :target
:result :result-summary :info}]
:timeline [{:kind :ref :turn-id :iteration-id :content :code
:status :duration-ms :result-summary}]
:turns
[{:id :user-request :status :prior-outcome :provider :model
:iteration-count :failure-count
:tokens :cost-usd :content
:iterations
[{:id :position :status :duration-ms
:provider :model :thinking :error
:tokens :cost-usd
:answer-position :returned-empty-blocks?
:vars
[{:name :code :value :version}]
:attachments
[{:id :source :tool-call-id :position :kind
:media-type :filename :size :stored}]
:blocks
[{:position :code :comment :result :stdout :error
:duration-ms :timeout? :repaired?}]}]}]}
The Markdown renderer renders thinking, iteration-level errors,
vars, per-block forensic previews, and final answer text. Large
fields are bounded so reports stay safe to open.Pre-turn "session_workspace" CTX block (STRING-KEYED — crosses the
Python boundary as session["workspace"]).
Sessions may work directly in trunk or inside an isolated backend
workspace. That distinction is reported on "sandbox", NOT
as a VCS. "vcs_kind" reports the underlying repository VCS ("git"
when the root is inside a git repo, else "none") so it matches the
git/ extension surface, which activates on the same predicate. The
model reads the workspace block to know the active root and what it has
changed since the fork. The block is stamped once per turn at engine
start; ctx_renderer serialises it verbatim.
Pre-turn `"session_workspace"` CTX block (STRING-KEYED — crosses the Python boundary as `session["workspace"]`). Sessions may work directly in trunk or inside an isolated backend workspace. That distinction is reported on `"sandbox"`, NOT as a VCS. `"vcs_kind"` reports the underlying repository VCS (`"git"` when the root is inside a git repo, else `"none"`) so it matches the `git/` extension surface, which activates on the same predicate. The model reads the workspace block to know the active root and what it has changed since the fork. The block is stamped once per turn at engine start; ctx_renderer serialises it verbatim.
Declarative /draft … slash tree.
Drafts are OPT-IN. By default a session works directly in the user's
real cwd (trunk). /draft new <label> clones cwd into an isolated
draft (an isolated workspace named <label>) and enters it; /draft apply
lands the draft's changes into cwd and leaves the draft; /draft abandon discards it and leaves. The header shows <label> (DRAFT)
while you're in one.
/draft show whether you're on trunk or in a draft /draft new <label> clone cwd into a draft named <label>, enter it /draft apply land the draft's changes into cwd, leave the draft /draft abandon [why] discard the draft, leave it /draft-blank <label> like /draft new, but the draft starts EMPTY — no files from the current HEAD are carried in
Filesystem (/cd) — session-scoped, every channel. What the jail ALLOWS is
derived from config (jail.filesystem in vis.yml, global or project); /cd
moves the session's PRIMARY LIVE root within that grant:
/cd [path] show / CHANGE the session's filesystem root
Vis owns no git lifecycle — apply copies the changed files into the
user's real cwd, uncommitted. Handlers are PURE w.r.t. the channel.
Declarative `/draft …` slash tree.
Drafts are OPT-IN. By default a session works directly in the user's
real cwd (trunk). `/draft new <label>` clones cwd into an isolated
draft (an isolated workspace named `<label>`) and enters it; `/draft apply`
lands the draft's changes into cwd and leaves the draft; `/draft
abandon` discards it and leaves. The header shows `<label> (DRAFT)`
while you're in one.
/draft show whether you're on trunk or in a draft
/draft new <label> clone cwd into a draft named <label>, enter it
/draft apply land the draft's changes into cwd, leave the draft
/draft abandon [why] discard the draft, leave it
/draft-blank <label> like /draft new, but the draft starts EMPTY —
no files from the current HEAD are carried in
Filesystem (`/cd`) — session-scoped, every channel. What the jail ALLOWS is
derived from config (`jail.filesystem` in vis.yml, global or project); `/cd`
moves the session's PRIMARY LIVE root within that grant:
/cd [path] show / CHANGE the session's filesystem root
Vis owns no git lifecycle — `apply` copies the changed files into the
user's real cwd, uncommitted. Handlers are PURE w.r.t. the channel.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |