com.blockether.vis.internal.env-python — com.blockether/vis 0.1.10

Embedded-GraalPy sandbox machinery — the agent's action substrate. The agent writes Python; this ns embeds a GraalPy org.graalvm.polyglot.Context, marshals values across the Clojure↔Python boundary, wires the Clojure tool fns into the Python globals as ProxyExecutables (so cat("x") in Python runs the Clojure cat), and runs the model's code block as ONE whole-block coroutine.

Public surface used by the loop:

create-python-context / set-python-binding! / bind-and-bump! / bind-and-bump-with-doc! / count-top-level-forms / validate-non-empty-block! / validate-no-banned-defs! / restore-sandbox! / SYSTEM_VAR_NAMES / system-var-sym? / lru-atom / current-turn-position / fresh-lru-atom / run-python-block / map-polyglot-error / bind-ctx! / ctx->python-str

The :python-context slot holds the GraalPy Context; the Python top scope is context.getBindings("python"). GraalPy ships in the default deps (runs on Oracle GraalVM 25 → Truffle gets the Graal JIT).

*auto-repair-brackets?* clj

When true, a bracket-balance syntax hint ALSO appends repair-bracket-balance's single-candidate suggested fix. OFF by default: the walker only DIAGNOSES; the auto-fix stays gated behind this flag until proven safe in the wild.

*current-turn-position* clj

*lru-atom* clj

->clj clj

(->clj v)

Polyglot Value (a Python value) -> Clojure data. STRINGS-ONLY boundary: dicts -> maps with VERBATIM STRING keys (exactly what Python held — no keywordizing, no regex key-shape sniffing), lists/tuples -> vectors, host objects (Java values that crossed the boundary, e.g. UUIDs) -> their underlying Java value via asHostObject, callables/opaque objects -> the raw Value. A non-string Python dict key (int, tuple, ...) stringifies via its Clojure conversion so the map stays string-keyed and total.

->py clj

(->py x)

Clojure value -> something GraalPy accepts as a Python value. STRINGS-ONLY boundary: map keys must be strings and no keyword/symbol may appear at any depth — a violation throws boundary-violation! naming the key path. Primitives and Strings pass through (the Context auto-converts Java boxed types); collections become polyglot proxies so Python sees dict/list; leaves convert via leaf->py (UUID/Temporal/Date -> ISO strings).

AUTO_IMPORTED_PYTHON_NAMES clj

Python names installed into builtins for every python_execution context. This is the model-facing inventory; keep it synchronized with auto-imports-python and its real-context regression test.

BANNED_DEF_HEADS clj

Python constructs refused pre-eval — belt-and-suspenders against the obvious sandbox-escape footguns on top of the Context restrictions.

bind-and-bump! clj

(bind-and-bump! env sym val)

Set sym -> val in the env's Python sandbox.

bind-and-bump-with-doc! clj

(bind-and-bump-with-doc! env sym doc val)

Like bind-and-bump! but also records doc in the side __vis_docs__ dict so a future live-vars view can surface name + doc (Python has no var metadata channel for doc text).

bind-ctx! clj

(bind-ctx! python-context data)

Bind the standing context in the sandbox as an ordered polyglot dict (->py → ProxyHashMap, which GraalPy treats as a NATIVE dict: session["k"], .get, .items(), comprehensions and {**ctx} all work) built from the SAME projection the renderer prints — so the live dict and the wire's session["a"]["b"] = … structural deltas agree. Bound under session only — decoupled from r, no legacy context alias. No JSON round-trip.

boundary-view clj

(boundary-view x)

(boundary-view x path)

What a plain-data Clojure value LOOKS LIKE after the GraalPy round trip — the mechanical composition of ->py then ->clj without a Python context. STRINGS-ONLY: string map keys pass VERBATIM, sets/seqs -> vectors, UUID/Temporal/Date leaves -> ISO strings. A keyword/symbol anywhere (key or value, any depth) throws boundary-violation! exactly like the real boundary — fix the producer fixture, never catch it. Idempotent.

Every tool result the model sees in production (serialized structurally by ctx-renderer/render-form-value) has already crossed this boundary, so assertions about what the model reads MUST be written against THIS shape. Tests feed (boundary-view raw-result) to pin that contract without booting GraalPy.

boundary-violation! clj

(boundary-violation! kind x path)

Throw on a keyword/symbol trying to cross the Clojure->Python boundary. The boundary is STRINGS-ONLY: every map that crosses (tool results, ctx, verb payloads) is built with string keys and string enum values at the SOURCE — there is no silent keyword->string conversion, so a keyword here means a producer bug, not data. path is the key path down from the value handed to ->py, so the offending producer field is nameable.

collect-garbage! clj

(collect-garbage! environment)

Best-effort GC between turns. Two steps, because GraalPy reclaims native-extension (numpy/pandas/PIL) memory in TWO stages:

1. guest gc.collect() runs the cycle detector, marking dead native cycles so their Java mirrors become weakly reachable, and
2. a JVM System.gc() then lets the Java tracing GC collect those mirrors and drain the reference queue that actually frees the native RSS (see graalpython IMPLEMENTATION_DETAILS: the guest collect ALONE does not free non-cyclic native objects whose only managed ref was just dropped). Runs while the interpreter is idle between turns, the cheapest time for a pause. Never throws; a closed/cancelled context is ignored.

count-top-level-forms clj

(count-top-level-forms python-context code)

Number of top-level Python statements in code, parsed inside the session's own GraalPy Context. Comment-/whitespace-only blocks return 0. The source is passed directly to a cached helper — no shared scratch global, auxiliary Context, or cross-thread race.

create-python-context clj

(create-python-context custom-bindings)

(create-python-context custom-bindings roots-fn)

(create-python-context custom-bindings roots-fn network-opts)

(create-python-context custom-bindings roots-fn network-opts stdin)

Create one persistent, deny-by-default GraalPy Context for a Vis session.

custom-bindings maps symbols to tool/verb functions or values. roots-fn optionally grants filesystem access confined to the current workspace roots. Every session Context rides the process-wide shared-engine; parsing and extension shims live inside that same Context. Rendering is pure JVM code, so normal sessions allocate no auxiliary GraalPy contexts. Extensions deliberately SHARE this one session Context (installed as guest callables, not separate contexts) — so a session holds exactly one Context; the only extra is a transient fork-context! child, created solely for sub_loop parallelism. The 4-arity stdin (optional InputStream) is wired to the guest sys.stdin — used by vis python to forward the caller's real stdin; agent sandboxes leave it nil.

ctx->python-str clj

(ctx->python-str data)

Render plain boundary data as a deterministic, executable Python literal.

This is deliberately a pure JVM serializer: rendering never enters GraalPy, never waits behind a session's GIL, and needs no process-global printer Context or lock. It mirrors the Clojure->Python boundary (string-only map keys, list-like collections, ISO strings for Date/UUID/Temporal) and keeps insertion order plus the historical 100-column layout.

eval-block clj

(eval-block python-context code)

Evaluate a whole Python code block in python-context. Returns {:source code :result <clj>} on success; throws the PolyglotException on failure (caller maps it to the engine error shape). Globals (defs/imports/ state) persist across calls in the same context.

fork-context! clj

(fork-context! custom-bindings)

(fork-context! custom-bindings roots-fn)

(fork-context! custom-bindings roots-fn network-opts)

Fork a CHILD agent Context for a sub_loop — same deny-by-default sandbox as the main context, built ON the shared Engine so it is SAFE to create even while the parent's eval is running (GraalVM-verified: no Truffle deadlock). custom-bindings wires the child's tool/verb fns, which close over the CHILD's env (its own ctx-atom). Returns the same {:python-context :sandbox-ns :initial-ns-keys} shape as create-python-context. The caller owns the child Context's lifecycle (close it when the sub_loop ends). roots-fn (optional) confines the child's Python filesystem to the current filesystem roots, same as the parent.

fresh-lru-atom clj

(fresh-lru-atom)

graal-resource-cache-redirected clj

map-polyglot-error clj

(map-polyglot-error python-context e code)

Map a GraalPy PolyglotException into the engine's op-error shape. :phase is :python/syntax for parse errors, else :python/runtime; :line/:column come from the Python source location when present. A host (Clojure-tool) exception is unwrapped so its real message surfaces. Recurring syntax-failure classes get an actionable hint prepended: a NON-ASCII char in code position (em-dash, x, curly quote - CPython's invalid character, precise wherever it lands), a PROSE-leading reply (see prose-leading-syntax-hint, first-line only), and - via parse-diagnose - an unbalanced double-quote or an unbalanced (), [], {} bracket pinpointed to its line/col.

normalize-dict-key clj

(normalize-dict-key s)

Model-input hygiene at the ONE inbound conversion: a dict key spelled ":from_anchor" is still a STRING (the model drifting into colon spelling while reading keyword-heavy source), so strip the single leading colon when an identifier char follows and the call just works — no lecture, no failure. Data keys are untouched: anchors ("44:f14") start with a digit, paths with a letter or /, neither with :. Produces strings, never keywords.

polyglot-noise-silenced clj

python-binding-names clj

(python-binding-names sym)

Canonical Python global plus intentional compatibility aliases for sym. Used by provider/native discovery to deduplicate the same capability.

remove-python-binding! clj

(remove-python-binding! python-context sym)

Remove sym from the Python sandbox globals ENTIRELY — the member key disappears, so apropos/dir no longer list it and calling it raises a plain NameError. This is how a deactivated tool must vanish: putMember nil only parks a None under the name, which apropos still lists and which calls as 'NoneType is not callable'.

restore-sandbox! clj

(restore-sandbox! _python-context _db-info _session-id)

NOOP. The session has ONE persistent interpreter; globals (defs/imports/vars) persist NATURALLY across turns, so there is nothing to restore.

run-python-block clj

(run-python-block python-context code & [_opts])

Evaluate one Python code block in python-context as ONE WHOLE-BLOCK coroutine, returning the FLAT sum-typed outcome:

{:stdout <printed>} ; SUCCESS — python_execution (what it print()ed) {:result <value>} ; SUCCESS — a native tool value (nothing printed) {:error <op-error>} ; FAILURE — the raised error IS the result

__vis_run_async__ AST-wraps the block in an async def, AUTO-SETTLES every bare top-level tool call (so cat(x) without await still runs), drives it as a single coroutine, and maps any raised exception against the WHOLE source. The program runs exactly as the model wrote it — Python's own halt-on-exception decides what ran. A pre-eval protected-rebind violation short-circuits to an :error instead.

seed-cli-runtime! clj

(seed-cli-runtime! python-context {:keys [argv env]})

Seed a standalone vis python CLI context with script argv (bound to sys.argv) and, when non-empty, an env map merged into os.environ. This is what gives the CLI real-python semantics: unlike the deny-by- default AGENT sandbox (env scrubbed for isolation — the human never sees their shell here), the human-run CLI forwards trailing script args and, by default, the caller's environment.

Mirrors the VIS_OUTBOX injection: values cross via putMember, then a guest eval assigns them (a JSON hop keeps ProxyHashMaps off the boundary and reuses the auto-imported json). Best-effort: a bad value never aborts startup.

set-advertised-native-tools! clj

(set-advertised-native-tools! python-context names)

Update names already visible in the provider's native tool schema. Native apropos suppresses these names; in-Python apropos() remains a complete, filterable sandbox index. Best-effort so discovery metadata cannot break eval.

set-python-binding! clj

(set-python-binding! python-context sym val)

Bind sym -> val in the Python sandbox globals. Clojure fns are wired as callables; everything else is marshalled.

ASYNC-BY-DEFAULT: a tool fn bound here is also DEFERRED (wrapped by __vis_deferred__, same as build-agent-context's defer step) so await tool(...) / gather(tool(...)) work. This matters because extension and foundation tools are (re)installed via this fn AFTER the context's own defer pass — without deferring here they'd stay raw/synchronous and the await the prompt teaches would fail. The compaction verbs (session_fold/session_drop/__vis_par__) are bound via create-python-context, not here, so they stay direct. No-op when the async preamble isn't installed (the printer/parser helper contexts never bind tools).

set-python-binding-doc! clj

(set-python-binding-doc! python-context sym doc)

Record doc text for sym (and its py-aliases) in the sandbox __vis_docs__ dict that in-sandbox doc(name) / apropos(pat) read. Keyed by the SAME py-name(s) set-python-binding! binds under, so doc("mcp_servers") resolves for ALIASED extensions that bind AFTER context creation (per turn, via sync-active-extension-symbols!) — not only the built-ins seeded eagerly in build-agent-context. No-op for a blank doc or a helper context with no sandbox.

shared-engine clj

sym->py-name clj

(sym->py-name sym)

Clojure tool/binding symbol -> a Python-LEGAL global name. Purely mechanical: / and - fold to _ (alias fold + kebab->snake); a trailing ! (mutation marker) is dropped; a trailing ? (predicate) becomes an is_ prefix. So git/status -> git_status, git/commit! -> git_commit, search/web -> search_web, file-exists -> file_exists. FULL SNAKE: this is how the agent reaches the tools — git_status() calls git/status.

A tiny compatibility alias layer may additionally expose selected historical short names (currently find_files/find for grep), but the snake name remains canonical.

system-var-sym? clj

(system-var-sym? sym)

SYSTEM_VAR_NAMES clj

Engine-owned symbols hidden from user live-var listings.

validate-no-banned-defs! clj

(validate-no-banned-defs! python-context code)

Throws :vis/banned-def-head when code references a banned construct (BANNED_DEF_HEADS). Parse failures are silent — the eval that follows surfaces a clean syntax error with line/column.

validate-non-empty-block! clj

(validate-non-empty-block! python-context code)

Throws :vis/empty-block when code parses to zero top-level statements (comment-only blocks). Iterations that produce no evidence are rejected at the model boundary.