Session environment lifecycle and the in-process session cache.
Creates and disposes a session environment (router, Python sandbox and
security snapshot), keeps Python extension symbols in sync, caches live
environments, reaps idle ones, applies /reload and provider changes to
cached sessions, and serializes turns per session.
Session environment lifecycle and the in-process session cache. Creates and disposes a session environment (router, Python sandbox and security snapshot), keeps Python extension symbols in sync, caches live environments, reaps idle ones, applies `/reload` and provider changes to cached sessions, and serializes turns per session.
(acquire-turn-lock! id)Take session id's one-turn-at-a-time lock and return the entry that owns
it, WITHOUT ever parking on it forever.
This used to be a bare .lock. A turn wedged inside the engine — parked on
CPython's GIL, where Thread.interrupt cannot reach it — never unlocks, so
every later turn for that session parked in Unsafe.park: turn.started on
the wire, not one event after it, and deaf to its own cancel, for the life of
the daemon. Meanwhile the cancel backstop had already synthesized
turn.cancelled and the daemon reported the session idle.
So the wait is a POLL. Each round re-reads the session's CURRENT entry, and a
CONDEMNED one is detached and rebuilt rather than waited on; tryLock is
interruptible, so a queued turn's own cancel finally reaches it.
A FREE lock is not enough. A session whose context was disposed — a teardown,
a recycle, an environment that failed halfway through being built — cannot be
entered again, and env-python/context-enterable? is what says so. A normal
cancel leaves the session standing; only a worker that fails to acknowledge or
unwind the interrupt is retired, and that process has already been killed.
Nothing is disposed here because parent-side host work may still hold the old environment. The expensive interpreter process is already reclaimed on the retirement path. Rescue happens at most once per acquisition — a fresh context that still refuses is a real turn failure, not a reason to keep minting workers.
Take session `id`'s one-turn-at-a-time lock and return the entry that owns it, WITHOUT ever parking on it forever. This used to be a bare `.lock`. A turn wedged inside the engine — parked on CPython's GIL, where `Thread.interrupt` cannot reach it — never unlocks, so every later turn for that session parked in `Unsafe.park`: `turn.started` on the wire, not one event after it, and deaf to its own cancel, for the life of the daemon. Meanwhile the cancel backstop had already synthesized `turn.cancelled` and the daemon reported the session idle. So the wait is a POLL. Each round re-reads the session's CURRENT entry, and a CONDEMNED one is detached and rebuilt rather than waited on; `tryLock` is interruptible, so a queued turn's own cancel finally reaches it. A FREE lock is not enough. A session whose context was disposed — a teardown, a recycle, an environment that failed halfway through being built — cannot be entered again, and `env-python/context-enterable?` is what says so. A normal cancel leaves the session standing; only a worker that fails to acknowledge or unwind the interrupt is retired, and that process has already been killed. Nothing is disposed here because parent-side host work may still hold the old environment. The expensive interpreter process is already reclaimed on the retirement path. Rescue happens at most once per acquisition — a fresh context that still refuses is a real turn failure, not a reason to keep minting workers.
In-process env cache.
Keyed by java.util.UUID session-soul-id. Under the 1:1 session ↔
workspace invariant this key is isomorphic to (:workspace/id env)
— one cache entry = one session = one workspace = one Python sandbox
lineage. Lookups normalize incoming strings to UUID via cache-key
so string-id callers keep working alongside the UUID key.
In-process env cache. Keyed by `java.util.UUID` session-soul-id. Under the 1:1 session ↔ workspace invariant this key is isomorphic to `(:workspace/id env)` — one cache entry = one session = one workspace = one Python sandbox lineage. Lookups normalize incoming strings to UUID via `cache-key` so string-id callers keep working alongside the UUID key.
(cache-env! session-id env)Insert env into the cache under session-id (UUID, or string
normalized via cache-key). Returns {:id <UUID> :environment env}.
Insert `env` into the cache under `session-id` (UUID, or string
normalized via `cache-key`). Returns `{:id <UUID> :environment env}`.(cache-key id)Normalize an id-shaped value (UUID or string-UUID) to a UUID
suitable for keying cache. Nil → nil so wrapped lookups stay
honest.
Normalize an id-shaped value (UUID or string-UUID) to a UUID suitable for keying `cache`. Nil → nil so wrapped lookups stay honest.
(condemn-env! id)Mark session id's engine entry CONDEMNED: the daemon has already declared
this session's turn over, but the thread that ran it never came back, so it
may be holding the entry's ReentrantLock forever.
The mark is a fact the NEXT turn reads: acquire-turn-lock! abandons a
condemned entry instead of queueing behind a dead lock. A worker that does
thaw clears the mark simply by taking the lock normally, so a backstop that
fired early costs nothing. Returns true when an entry was marked.
Mark session `id`'s engine entry CONDEMNED: the daemon has already declared this session's turn over, but the thread that ran it never came back, so it may be holding the entry's `ReentrantLock` forever. The mark is a fact the NEXT turn reads: [[acquire-turn-lock!]] abandons a condemned entry instead of queueing behind a dead lock. A worker that does thaw clears the mark simply by taking the lock normally, so a backstop that fired early costs nothing. Returns true when an entry was marked.
(create-environment router
{:keys [db session channel external-id title workspace-id]})Creates a vis environment (component) for session lifecycle and querying.
The environment holds:
Params:
router - Required. Result of llm/make-router.
opts - Map with :db and optional :session,
:channel, :external-id, :title.
:db accepted forms:
nil - no DB (sandbox-only execution)
:memory - ephemeral in-process SQLite DB
path string - persistent SQLite DB at path
{:path p} - persistent SQLite DB at path
{:datasource ds} - caller-owned DataSource (not closed on dispose)
Returns the vis environment map.
Creates a vis environment (component) for session lifecycle and
querying.
The environment holds:
- Python sandbox context with custom bindings + bindings cache
- DB connection (or shared-mem datasource)
- Router (LLM provider config)
- Extension registry atom
Params:
`router` - Required. Result of `llm/make-router`.
`opts` - Map with `:db` and optional `:session`,
`:channel`, `:external-id`, `:title`.
`:db` accepted forms:
nil - no DB (sandbox-only execution)
:memory - ephemeral in-process SQLite DB
path string - persistent SQLite DB at path
{:path p} - persistent SQLite DB at path
{:datasource ds} - caller-owned DataSource (not closed on dispose)
Returns the vis environment map.(dispose-environment! environment)Disposes a vis environment and releases resources. For persistent DBs
(created with :path), data is preserved. For disposable DBs, all
data is deleted.
Every env owns its DB connection, so disposing one always closes it.
Disposes a vis environment and releases resources. For persistent DBs (created with `:path`), data is preserved. For disposable DBs, all data is deleted. Every env owns its DB connection, so disposing one always closes it.
(dispose-reloaded-sandbox! k)Close an idle, reload-stale sandbox without rebuilding it. The cached env and its lock remain until the next turn rebuilds the policy. Recheck ownership under the lock so a displaced entry cannot close another turn's sandbox.
Close an idle, reload-stale sandbox without rebuilding it. The cached env and its lock remain until the next turn rebuilds the policy. Recheck ownership under the lock so a displaced entry cannot close another turn's sandbox.
(gateway-runtime-metrics)Bounded process/runtime gauges for the gateway metrics endpoint. Values are sampled on demand; no profiler or background allocation is required.
Bounded process/runtime gauges for the gateway metrics endpoint. Values are sampled on demand; no profiler or background allocation is required.
(install-extension! environment ext)Register a validated extension into environment (per-env registration,
distinct from the global-registry register-extension! defined earlier
in this file).
If an extension with the same :ext/name is already registered,
it is replaced (not duplicated). Enables hot-swap via
reload-extension! (removed for GraalVM native-image compatibility).
Returns environment for chaining.
Register a validated extension into `environment` (per-env registration, distinct from the global-registry `register-extension!` defined earlier in this file). If an extension with the same `:ext/name` is already registered, it is replaced (not duplicated). Enables hot-swap via `reload-extension!` (removed for GraalVM native-image compatibility). Returns `environment` for chaining.
(mark-policy-reload!)Invalidate every cached env's policy and close idle Python workers now. Busy sessions close their stale sandbox after the current turn releases its lock. The next turn rebuilds the environment from the reloaded configuration.
Invalidate every cached env's policy and close idle Python workers now. Busy sessions close their stale sandbox after the current turn releases its lock. The next turn rebuilds the environment from the reloaded configuration.
(open-env! id {:keys [channel external-id title workspace-id]})Open or resume a session with its project bound before resolving config and providers.
Open or resume a session with its project bound before resolving config and providers.
(reap-idle-envs!)One reaper sweep: dispose + evict cached session envs idle past
env-idle-ttl-ms (or, under memory pressure past env-rss-budget-mb,
EVERY idle env this sweep — TTL ignored), then — if the cache still exceeds
env-cache-max — force-evict the least-recently-active idle entries until
back under the cap. Every eviction is lock-guarded (a running turn is
skipped). Returns the number of entries evicted. Safe to call directly
(tests / manual sweeps).
One reaper sweep: dispose + evict cached session envs idle past `env-idle-ttl-ms` (or, under memory pressure past `env-rss-budget-mb`, EVERY idle env this sweep — TTL ignored), then — if the cache still exceeds `env-cache-max` — force-evict the least-recently-active idle entries until back under the cap. Every eviction is lock-guarded (a running turn is skipped). Returns the number of entries evicted. Safe to call directly (tests / manual sweeps).
(recycle-env! k)/reload recycle: rebuild a FRESH env for session k under the new security
policy and swap it into the existing cache entry IN PLACE — REUSING the same
ReentrantLock so a caller queued on the lock re-reads the fresh env — then
dispose the OLD Python session (and its own per-env DB connection). MUST be
called while holding the entry lock, so no turn races the swap and old is
stable. The transcript lives in the DB; open-env! resumes it, and the
session snapshot restores the helpers and variables the old sandbox saved.
`/reload` recycle: rebuild a FRESH env for session `k` under the new security policy and swap it into the existing cache entry IN PLACE — REUSING the same `ReentrantLock` so a caller queued on the lock re-reads the fresh env — then dispose the OLD Python session (and its own per-env DB connection). MUST be called while holding the entry lock, so no turn races the swap and `old` is stable. The transcript lives in the DB; `open-env!` resumes it, and the session snapshot restores the helpers and variables the old sandbox saved.
(refresh-cached-routers! router)Reseat :router on every cached env's environment map.
create-environment snapshots the router into
(:router env) at construction time, and the iteration loop calls
(svar/ask-code! (:router environment) ...) - not the global
router-atom. So when a frontend changes provider
config and rebuilds the global router, every long-lived env in the
cache (TUI keeps one for the whole session) keeps talking to the
previous model until disposed.
Provider kickoff hooks run against each session before its new snapshot is
seated, covering providers added or reconfigured while that session is live.
They run outside the cache swap, once per environment: a session replaced
while its kickoff ran is kicked off again on its new environment, and an
evicted one is dropped. A session whose kickoff fails keeps its previous
environment while every other session moves; the failures are then thrown as
one ex-info naming the affected session ids, with the first failure as cause.
Call this immediately after rebuild-router! so the next send! on any cached
session picks up the new router.
Reseat `:router` on every cached env's environment map. `create-environment` snapshots the router into `(:router env)` at construction time, and the iteration loop calls `(svar/ask-code! (:router environment) ...)` - not the global `router-atom`. So when a frontend changes provider config and rebuilds the global router, every long-lived env in the cache (TUI keeps one for the whole session) keeps talking to the *previous* model until disposed. Provider kickoff hooks run against each session before its new snapshot is seated, covering providers added or reconfigured while that session is live. They run outside the cache swap, once per environment: a session replaced while its kickoff ran is kicked off again on its new environment, and an evicted one is dropped. A session whose kickoff fails keeps its previous environment while every other session moves; the failures are then thrown as one ex-info naming the affected session ids, with the first failure as cause. Call this immediately after `rebuild-router!` so the next `send!` on any cached session picks up the new router.
(reload-router!)Rebuild the shared LLM router from the freshly reloaded config and reseat it
on every cached env. Registered as a /reload hook.
reload-slash re-reads vis.yml through config/reload-config!, but the
router is an immutable SNAPSHOT: built once by get-router and captured
again inside every long-lived session env ((:router environment)). Without
this hook a default_model / provider edit only took effect after a full
restart — the engine kept routing turns through the previous router, and
every frontend that names the router default (the TUI footer model chip via
resolve-effective-model) kept showing the OLD model.
No-ops when the router was never built, so lazy first use is preserved: a
/reload must not force OAuth token fetches at TUI boot. Returns nil.
Rebuild the shared LLM router from the freshly reloaded config and reseat it on every cached env. Registered as a `/reload` hook. `reload-slash` re-reads vis.yml through `config/reload-config!`, but the router is an immutable SNAPSHOT: built once by `get-router` and captured again inside every long-lived session env (`(:router environment)`). Without this hook a `default_model` / provider edit only took effect after a full restart — the engine kept routing turns through the previous router, and every frontend that names the router default (the TUI footer model chip via `resolve-effective-model`) kept showing the OLD model. No-ops when the router was never built, so lazy first use is preserved: a `/reload` must not force OAuth token fetches at TUI boot. Returns nil.
(set-provider! provider)Set the single active provider config. Persists to disk, updates
in-memory state, rebuilds the global router, and reseats cached
session envs. provider is a svar-native provider map
{:id :base-url :api-key :models [...]}. Replaces an existing
provider with the same :id or appends a new entry.
Set the single active provider config. Persists to disk, updates
in-memory state, rebuilds the global router, and reseats cached
session envs. `provider` is a svar-native provider map
`{:id :base-url :api-key :models [...]}`. Replaces an existing
provider with the same `:id` or appends a new entry.(sync-active-extension-symbols! environment)(sync-active-extension-symbols! environment active-extensions)Make the Python sandbox's callable globals match active extension state.
install-extension! keeps every extension row in :extensions, but only
active extensions contribute callable symbols. Called after per-env
installation and again at turn start so :ext/activation-fn changes become
real tool availability, not just prompt visibility.
The Python sandbox is FLAT globals (no namespaces/aliases/macros): active extensions putMember their symbols straight into the top scope; deactivated extensions have theirs removed (putMember nil). Symbol names are snake-ified by env/set-python-binding!.
Make the Python sandbox's callable globals match active extension state. `install-extension!` keeps every extension row in `:extensions`, but only active extensions contribute callable symbols. Called after per-env installation and again at turn start so `:ext/activation-fn` changes become real tool availability, not just prompt visibility. The Python sandbox is FLAT globals (no namespaces/aliases/macros): active extensions putMember their symbols straight into the top scope; deactivated extensions have theirs removed (putMember nil). Symbol names are snake-ified by env/set-python-binding!.
(sync-cached-extension-symbols!)Synchronize extension bindings in every idle cached session immediately.
A Settings change is process-wide while each session owns a persistent Python context. Busy contexts retain their started tool surface and are synchronized at the next turn boundary. Returns the refreshed count.
Synchronize extension bindings in every idle cached session immediately. A Settings change is process-wide while each session owns a persistent Python context. Busy contexts retain their started tool surface and are synchronized at the next turn boundary. Returns the refreshed count.
(sync-extension-symbols-into! python-context environment active-extensions)The symbol sync itself, against a context handed in EXPLICITLY.
Separate from sync-active-extension-symbols! for one caller: the delay
that builds a session's sandbox runs this with its own fresh context. That
caller cannot reach the context through environment, because doing so
would re-enter the very delay it is running inside, and a Clojure delay
answers re-entry with a deadlock — the session's first turn would hang.
The symbol sync itself, against a context handed in EXPLICITLY. Separate from [[sync-active-extension-symbols!]] for one caller: the delay that builds a session's sandbox runs this with its own fresh context. That caller cannot reach the context through `environment`, because doing so would re-enter the very delay it is running inside, and a Clojure delay answers re-entry with a deadlock — the session's first turn would hang.
(touch-entry! entry)Bump entry's :last-active stamp to now so the reaper treats it as warm.
Returns entry for threading.
Bump `entry`'s `:last-active` stamp to now so the reaper treats it as warm. Returns `entry` for threading.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |