fractal.engine.api is the supported Clojure SDK facade. It is the public
embedding surface for starting sessions, running turns, reading durable state,
subscribing to live events, hydrating payloads, and reopening durable sessions.
The facade is intentionally small. Model-facing host functions are injected into session runtimes; they are not ordinary vars exported from this namespace.
(require '[fractal.engine.api :as fe])
flowchart LR
CFG["make-config"] --> START["start-session!"]
CFG --> RESUME["resume-session!"]
START --> RUN["run-turn! / run-turn-async!"]
RESUME --> RUN
RUN --> READS["view / progress / events-since / read-payload"]
RUN --> COMPACT["compact-session!"]
COMPACT --> READS
START --> STOP["stop-session!"]
RESUME --> CLOSE["close-session!"]
RUN --> CLOSE
STOP --> CLOSE
Exported functions:
(fe/make-config opts)
(fe/start-session! cfg)
(fe/start-session! cfg opts)
(fe/run-turn! handle msg)
(fe/run-turn-async! handle msg)
(fe/run-turn-with-contract! handle msg contract)
(fe/resume-session! cfg sid)
(fe/resume-session! cfg sid opts)
(fe/fork-session! cfg source-sid) ; 0.7, alpha
(fe/fork-session! cfg source-sid opts)
(fe/stop-session! handle)
(fe/stop-session! handle opts)
(fe/close-session! handle)
(fe/compact-session! handle)
(fe/view handle)
(fe/progress handle)
(fe/event-stream handle)
(fe/events-since handle event-id)
(fe/read-payload handle ref-or-value)
(fe/subscribe! handle callback)
;; embedder facts, pins, projections (0.6)
(fe/append-fact! handle fact)
(fe/facts-since handle fact-id)
(fe/pin! handle pin)
(fe/read-pin handle pin-name)
(fe/list-pins handle)
(fe/delegation-report handle turn-id)
(fe/verify-claim source claim)
;; recorded replay (0.6)
(fe/replay-responder source sids)
(fe/surface-calls source sid)
(fe/open-sqlite-store dir)
(fe/close-store! store)
(fe/responder clauses)
responder is a fake-adapter helper for tests and local examples.
event-stream serves the session's FULL ordered event log from the durable
store. After a durable reopen the in-process view holds a bounded working
window of events (see snapshot reopen in
Storage And Heads); the log remains the truth and this
function reads the log.
The command-line facade is fractal.engine.cli, exposed by the :cli alias:
clojure -M:cli <command> [options] [args]
It is a thin process-per-command wrapper over this API. The CLI should not be
used as a different semantic layer: run and turn call run-turn!, resume
reopens through resume-session!, compact calls compact-session!, readback
commands call view, progress, events-since, or read-payload, and config
resolution ends in make-config.
The CLI defaults to JSON for agent automation and supports --edn when callers
need exact Clojure values such as payload refs. Its full command contract is in
Agent Control Plane.
make-config normalizes and validates a config map. It records adapter choice,
resolves the capability profile, resolves known model context windows, applies
defaults, and validates enumerated options.
Required or notable options:
{:adapter :fake | :sdk
:model "model-id"
:provider :provider-id ; optional explicit provider override
:provider/config {...} ; optional, provider-specific SDK config
:fake/respond respond-fn ; required when :adapter is :fake
:capability :locked-down | :default | :trusted | profile-map
:harness :clojure | :rlm
:leaf-model "model-id" | nil ; defaults to root :model
:leaf-provider :provider-id | nil ; defaults to resolved root provider
:child-model "model-id" | nil ; defaults to root :model
:child-provider :provider-id | nil ; defaults to resolved root provider
:store :memory | :sqlite
:store/dir "relative-or-configured-dir"
:max-steps 25
:max-turns nil
:call-timeout-ms 120000
:max-fanout 50
:fanout-pool 16
:leaf-concurrency 8
:retry true
:stream? false
:cache-ttl "1h" ; "5m" or "1h"
:live/queue-bound 1024
:live/drop :drop-transient
:context {:compact-at 0.80
:hard-at 0.95
:unknown-window-chars 400000}
:observe {:ok-fit 400 :final-fit 1200} ; fit-or-stub observation caps (chars)
:doctrine nil ; stamped base-doctrine override, see below
:surface/record? false ; record surface calls as durable events
:surface/replay-calls nil ; recorded calls to serve instead of live fns
:writer-lease-ttl-ms nil ; sqlite writer-lease staleness window (default 10 min)
:writer-lease-steal? false ; explicit crashed-writer lease takeover
:system-overlay nil
:surfaces []}
:doctrine injects embedder-owned base doctrine text in place of the
harness-selected built-in prompt, stamped with the engine's name/version/hash
discipline and inherited by child sessions:
:doctrine {:doctrine/name :myapp/frame
:doctrine/version 3
:doctrine/text "You are ..."}
The stamp (never the text) is part of the session's recorded bundle identity, so resume verifies the session reopens under the doctrine it ran with.
:observe sets the observation window — how many characters of a return value
the model sees whole before it gets a «type, size» stub and must inspect or
slice deliberately. Narrow windows enforce a blind-root discipline; the
defaults match prior releases.
make-config throws ex-info for invalid config, including:
:config/invalid-cache-ttl
:config/invalid-adapter
:config/missing-responder
:config/missing-model
:config/invalid-harness
:config/invalid-store
:config/missing-store-dir
:config/invalid-doctrine
:config/invalid-observe
:config/invalid-surface-replay
:capability/unknown-profile
:capability/invalid
The default harness is :clojure. Use :rlm only when the model should receive
the recursive host-function surface described below.
:capability selects the SCI profile used for model-written Clojure:
:locked-down injects only FINAL / inspect and denies filesystem, shell,
network, Java interop, and recursive model egress.:default allows read-only access inside the work area and a small read-only
shell command allowlist; writes, network, Java interop, and dangerous classes
remain denied.:trusted is a local trusted-use profile that intentionally opens work-area
writes, shell, and network.Custom profiles are validated before use. Per-session and child profiles are clamped against their parent profile; an override can narrow access but cannot widen it.
Surface functions are a separate finite gate:
:surface/fns '#{jira/search git/status git/diff}
The default is deny. A function appears in SCI only when its surface is configured and the resolved capability profile allows its qualified symbol. Child sessions inherit surface functions through set intersection, so a child cannot gain a world/API function its parent did not have.
Runtime governor keys bound live and recursive work:
:max-steps produces :budget-exceeded when one turn consumes too many loop
iterations.:max-turns throws :fractal/session-turn-limit before opening another turn.:call-timeout-ms produces :timeout when an adapter call and its retry loop
exceed the wall-clock deadline.:max-fanout rejects over-wide map-lm / map-rlm inputs.:fanout-pool bounds worker threads for one fan-out.:leaf-concurrency bounds concurrent leaf calls across the process.:context {:hard-at ...} produces :budget-exceeded before hard context
exhaustion.These are execution controls. TurnResult includes provider usage/cost/cache
records when known, and those records are observability metadata for audit and
reporting.
The fake adapter runs fully offline and is the simplest way to test API usage.
(require '[fractal.engine.api :as fe])
(def cfg
(fe/make-config
{:adapter :fake
:model "fake-model"
:capability :default
:fake/respond
(fe/responder
[[:default "```clojure\n(FINAL {:answer 42})\n```"]])}))
(def handle (fe/start-session! cfg {:id "example-session"}))
(try
(let [result (fe/run-turn! handle "What is 6 times 7?")]
(:turn/final-value result))
(finally
(fe/stop-session! handle)))
;; => {:answer 42}
fe/responder accepts [[match reply] ...] clauses. match can be a substring
of the last user message, a predicate on the provider request, or :default.
reply can be an assistant string, a call-record map, or a function of the
provider request.
The live adapter is :sdk. Provider credentials are not part of this repository
surface. Pass provider config from your runtime environment or omit
:provider/config and let the SDK use its own defaults.
(require '[fractal.engine.api :as fe])
(def provider-config
(cond-> {}
(System/getenv "MODEL_API_KEY")
(assoc :api-key (System/getenv "MODEL_API_KEY"))))
(def cfg
(fe/make-config
{:adapter :sdk
:provider :your-provider
:model "provider-model-id"
:capability :default
:provider/config provider-config}))
(def handle (fe/start-session! cfg))
If :provider is omitted, the engine attempts to resolve a provider from the
model catalog. If the model is unknown and no explicit provider is supplied,
session start throws :config/unknown-model.
Do not log or commit credential values. Treat :provider/config as runtime
configuration and keep public examples placeholder-only.
Use :store :sqlite with :store/dir to persist the session event log and
content-addressed payloads. Durable sessions can be closed and reopened with
resume-session!.
(require '[fractal.engine.api :as fe])
(def session-id "durable-example")
(def cfg
(fe/make-config
{:adapter :fake
:model "fake-model"
:capability :default
:store :sqlite
:store/dir "var/example-session-store"
:fake/respond
(fe/responder
[["define" "```clojure\n(def remembered 7)\n(FINAL :defined)\n```"]
["use" "```clojure\n(FINAL (* remembered 6))\n```"]])}))
(let [h1 (fe/start-session! cfg {:id session-id})]
(try
(fe/run-turn! h1 "define the value")
(finally
(fe/close-session! h1))))
(let [h2 (fe/resume-session! cfg session-id)]
(try
(:turn/final-value (fe/run-turn! h2 "use the value"))
(finally
(fe/close-session! h2))))
;; => 42
resume-session! is public but marked alpha. It currently requires
:store :sqlite; using it with :store :memory throws
:config/unsupported-store. Resuming an unknown session id throws
:fractal/unknown-session. Resume restores from the latest NON-aborted head —
a :turn-aborted wreckage head left by a failed turn is never the resume
basis.
fork-session! materializes a FRESH session from a selected immutable head of
a persisted :store :sqlite session — REPL vars restored, the source never
advanced. It is the host-side counterpart of the model-facing attach-rlm.
;; fork the latest non-aborted head:
(def fork (fe/fork-session! cfg source-session-id))
;; fork a specific head — the only way to reach a :turn-aborted wreckage head:
(def recovery (fe/fork-session! cfg source-session-id
{:head/id (:turn/aborted-head failed-result)}))
(fe/run-turn! fork "compute over the restored vars …")
(fe/close-session! fork)
opts: :head/id (explicit source head), :id (fork session id),
:capability (clamp-only override), :system-overlay,
:bundle/allow-mismatch?.
Typed failures: :config/unsupported-store (non-sqlite cfg),
:fractal/unknown-session, :fractal/unknown-head,
:fractal/fork-capability-rejected (the source ran more privileged than the
caller's cfg), :bundle/surface-mismatch (the fork did not re-present the
source's surfaces; :bundle/allow-mismatch? true is the explicit escape).
Sessions record a BUNDLE identity — the content-addressed stamp of
{harness, doctrine, surfaces, capability} they ran under — on the session
row and on every published head (:head/bundle-hash). Resume verifies the
configured world against it: a surface or doctrine mismatch throws a typed
:bundle/surface-mismatch / :bundle/doctrine-mismatch instead of silently
reopening the session in a different world. An intentional divergence (for
example resuming across a doctrine upgrade) is an explicit opt-in:
(fe/resume-session! cfg sid {:bundle/allow-mismatch? true})
Capability is NOT part of this equality check — it follows the clamp lattice
(a resumed session is clamped to what it ran with; narrowing stays legal).
Sessions recorded before bundles existed fall back to the legacy surface-stamp
check (:surface/mismatch).
run-turn! is blocking:
(fe/run-turn! handle "user message")
It returns a TurnResult when a turn opens and the runtime reaches a modeled
terminal outcome. It also returns a TurnResult when the session is already
stopped and no new turn is opened.
{:status :final | :error | :timeout | :budget-exceeded
:session/id "session-id"
:turn/id 1
:turn/final-value {:answer 42} ; present only when :status is :final
:turn/aborted-head "sha256:..." ; 0.7 — present on non-:final terminals
:turn/usage {:usage/status :known | :unknown, ...}
:turn/cost {:cost/status :known | :unknown, ...}
:turn/cache {:cache/status :hit | :miss | :unknown, ...}
:step-count 1
:error nil | {:error/type keyword, :error/message string, ...}}
Status meanings:
:final: the model called FINAL.:timeout: the adapter call exceeded :call-timeout-ms.:budget-exceeded: max steps or hard context-window limit stopped the turn.:error: provider failure, model/runtime error, stopped session, or another
non-final terminal failure.Every non-:final terminal turn additionally publishes a best-effort
:turn-aborted wreckage head and reports its id as :turn/aborted-head —
the aborted turn's accumulated REPL state stays recoverable via
fork-session!/attach-rlm with that explicit head id. Wreckage never
becomes a default resume/attach basis. See
STORAGE_AND_HEADS.md.
run-turn! throws for pre-turn failures, including:
:fractal/session-turn-limit:fractal/turn-in-flight:subscribe/reentrantA FAILED auto-compaction provider call no longer throws here (0.7): compaction
is skipped, a typed :compaction/failed event is appended, and the turn
proceeds — the hard context-window gate still bounds it.
run-turn-async! opens a turn synchronously and runs the step loop on a
background future:
(def async-result (fe/run-turn-async! handle "user message"))
(:turn/id async-result)
;; => 2
@(:promise async-result)
;; => TurnResult
Caller-thread behavior:
{:turn/id nil :promise p} where p is already
delivered with an error TurnResult:max-turns, :turn-in-flight, reentrant writes, and pre-open compaction
failures throw synchronouslyPromise behavior:
TurnResultrun-turn-with-contract! wraps run-turn! in a validate→correct→retry loop —
the pattern every embedder otherwise hand-rolls. The engine interprets
nothing: the contract is caller-supplied.
(fe/run-turn-with-contract! handle "produce the report"
{:validate (fn [turn-result]
(when-not (:sections (:turn/final-value turn-result))
"missing :sections")) ; nil/false = accept
:max-attempts 3 ; total turns spent
:correction (fn [rejection result] ...)}) ; optional; default restates
:validate runs only on :final results; returning nil/false accepts.
Any truthy value is the rejection fed into the correction message.:contract/rejected and
:contract/attempts attached — never swallowed.:final terminals (timeout/budget/error) return immediately; a
correction message cannot fix a dead turn.The durable store carries a STORE-scoped embedder layer so applications do not need a shadow database for state that is about engine facts:
(fe/append-fact! handle {:fact/tag :run/scored :fact/value {...}})
(fe/facts-since handle 0)
(fe/pin! handle {:pin/name :incumbent
:pin/ref {:session/id sid :head/id head-id}
:pin/meta {:why "current best"}
:pin/expected-version 2}) ; optional CAS (nil ⇒ must not exist)
(fe/read-pin handle :incumbent)
(fe/list-pins handle)
:fact/tag plus any EDN :fact/value. The
engine orders, persists, and serves them (facts-since is the stream app
projections fold from) and never interprets them — no schemas, no queries.:fractal/pin-cas-failed on a stale :pin/expected-version).{:session/id … :head/id …} pin ref must name a published head
(:fractal/dangling-ref otherwise). Values are EDN-coerced at write, so a
non-EDN member becomes an opaque marker instead of poisoning later reads.Read projections answer the questions embedders otherwise scrape the log for:
(fe/delegation-report handle turn-id)
;; => {:turn/self {...} :delegation/edges [...]
;; :delegation/children [{:child/session-id ... :child/status ...
;; :child/usage ... :child/cost ...}]
;; :delegation/children-cost {...}} ; :unknown-aware, never fabricated
(fe/verify-claim handle {:session/id sid :head/id head-id})
;; => {:claim/kind :head :verified? true}
verify-claim accepts a payload ref, {:session/id … :head/id …},
{:session/id … :edge/id …}, or {:session/id …} — mechanical existence
checks against the log for anything a model (or anyone) asserts exists.
delegation-report re-folds child logs (O(subtree log) on sqlite); treat it
as an inspection read and poll politely.
Every provider step, every leaf call, and (opt-in) every surface call is in the durable log — so a recorded run can re-execute deterministically with no provider spend:
;; record: any run against a sqlite store (leaf calls are always recorded;
;; surface calls require :surface/record? true in the recording cfg)
;; replay provider + leaf calls:
(def respond (fe/replay-responder source ["root-sid" "child-sid"]))
(def replay-cfg (fe/make-config {:adapter :fake :fake/respond respond
:model "fake-model" :harness :rlm}))
;; replay world reads too:
(def calls (into (fe/surface-calls source "root-sid")
(fe/surface-calls source "child-sid")))
;; ... :surface/replay-calls calls in the replay cfg; live fns are never invoked
;; cross-run sources:
(def store (fe/open-sqlite-store "var/recorded-run"))
;; ... (fe/close-store! store)
(fn, args). Root and
rlm children replay as a tree.:fractal/replay-*,
:surface/replay-*) — a replay never silently invents a response.stop-session! requests a stop:
(fe/stop-session! handle)
(fe/stop-session! handle {:wait? true})
It appends :session/stop-requested. If no turn is running, it also appends
:session/stopped. With :wait? true, it waits for the turn lock before
ensuring the stopped state.
close-session! releases process-local resources:
(fe/close-session! handle)
For SQLite stores, this closes the JDBC connection and stops dispatchers. A
closed durable session can be reopened with resume-session!.
compact-session! forces compaction immediately:
(fe/compact-session! handle)
It acquires the turn lock and throws :fractal/turn-in-flight if another turn
is running. Compaction can make a provider call and appends durable compaction
events when successful.
Read functions do not make provider calls:
(fe/view handle)
(fe/progress handle)
(fe/event-stream handle)
(fe/events-since handle event-id)
(fe/read-payload handle ref-or-value)
(fe/subscribe! handle callback)
view returns the strong folded session view from the store. It may contain
payload refs for large messages, final values, eval records, or snapshots.
progress returns a small polling shape:
{:session/id "session-id"
:session/status :running | :stop-requested | :stopped | :error
:running? true
:turn-count 1
:current-turn 1
:step-count 1
:in-flight false
:last-event-id 8}
event-stream returns the session's full ordered event log from the durable
store (after a snapshot reopen the in-process view's :events is a bounded
working window — the log stays the truth). events-since returns ordered
durable events with :event/id greater than the supplied id.
read-payload is the supported hydration seam:
(fe/read-payload handle maybe-ref)
It dereferences payload refs and returns non-ref values unchanged.
subscribe! registers a callback for durable events and transient deltas:
(def unsubscribe
(fe/subscribe! handle
(fn [event]
(println (:event/type event)))))
(unsubscribe)
A subscriber callback must not write to the same session. Reentrant writes throw
:subscribe/reentrant. Subscribers should use events-since to recover from
:subscribe/gap notifications.
The engine has two harness modes:
:clojure: default harness. The model-facing runtime includes FINAL and
inspection support.:rlm: recursive harness. The runtime also injects lm, map-lm, rlm,
map-rlm, and attach-rlm, subject to the capability profile.The model-facing functions are:
FINAL
lm
map-lm
rlm
map-rlm
attach-rlm
They are available inside the session's evaluated Clojure when the selected harness and capability profile allow them. They are not exported as public Clojure API functions.
In :rlm mode:
lm performs one bounded leaf provider call.map-lm performs ordered fan-out over leaf calls, capped by :max-fanout.rlm starts a fresh child session in the same store and runs it to FINAL.map-rlm performs ordered fan-out over child sessions, capped by
:max-fanout.attach-rlm starts a fresh derived child from a selected prior session/head;
the source session is not advanced.Child calls return envelopes containing :rlm/value, :rlm/session,
:rlm/head, and :rlm/meta. Partial fan-out failures are represented in-place
as sentinel maps such as {:fractal/failed true, ...} rather than aborting the
whole batch.
Root turn usage and cost stay scoped to the root turn. Child accounting is reported in the child envelope metadata.
:surfaces is an SDK extension point for host-provided worlds such as Git,
Jira, repository search, archives, or MCP-backed APIs. A descriptor supplies
namespaced functions plus prompt metadata:
{:surface/id :jira
:surface/version 1
:surface/prompt "Use jira/search when you do not know the issue key."
:surface/prompts
{:system "Stable Jira usage doctrine."
:request (fn [ctx] "Dynamic request-local Jira context.")
:leaf "Leaf calls classify Jira snippets only."}
:surface/namespaces
{'jira {'search {:doc "Search issues."
:arglists '([query opts])
:fn (fn [query opts] ...)}
'issue {:doc "Fetch one issue."
:arglists '([key opts])
:factory (fn [ctx] (fn [key opts] ...))}}}}
Rules:
(jira/search "auth" {:limit 10}).clojure.*, java.*, sci.*, and
fractal.engine.* are rejected.:surface/fns is deny-by-default and is clamped for children.:surface/prompt and :surface/prompts :system text renders into the
generated system surface card.:surface/prompts :request renders per root/child provider request
as transient prompt context; it is not appended to durable transcript state.:surface/prompts :leaf renders into lm / map-lm leaf system prompts.:bundle/surface-mismatch
(with {:bundle/allow-mismatch? true} as the explicit escape); legacy
sessions throw :surface/mismatch. On attach-rlm, the derived child must
re-present the SOURCE session's surfaces (its restored vars were created
against them) or attach fails typed, with the same explicit opt-out.Surface functions are trusted embedder code. The engine owns injection, gating, prompt truth, cache placement, and surface identity; embedders own auth, side-effects, rate limits, audit, and domain correctness.
This is an in-process SDK extension point, not a generic CLI plugin loader. A plain EDN CLI config file can reference normal data, but it cannot construct function objects by itself. Products that want CLI-accessible worlds should load their surfaces in process and then call the public API or provide their own thin CLI wrapper around that configured engine.
Each function entry contains exactly one callable source:
{:fn (fn [arg opts] ...)}
{:factory (fn [ctx]
(fn [arg opts] ...))}
Factories are useful when a surface needs per-session state. Factory context contains:
{:handle handle
:session/id "session-id"
:cfg cfg
:capability resolved-profile
:surface/id :jira
:surface/version 1
:surface/function 'jira/search}
Factories must return functions. Function objects are never included in public surface stamps or durable session metadata.
:surface/prompts may contain :system, :request, and :leaf. Each value is
either a string, a function returning nil/string, or nil.
Prompt function context always includes:
{:surface/id :jira
:surface/version 1
:surface/functions ['jira/search 'jira/issue]
:surface/prompt-phase :request}
Request prompt functions additionally receive the current :handle, :cfg,
:view, :session/id, :turn/id, and :step/id. Leaf prompt functions
receive :handle, :cfg, :session/id, :input, :query, and :mode.
Only capability-exposed surfaces can contribute prompt text in any phase.
Prompt order for root and child requests is:
:system-overlay;:system-overlay;Dynamic request context is deliberately outside durable message state. When it
is present, request cache metadata includes :breakpoints 1, allowing providers
with system-and-tail caching to keep the stable system prefix without treating
the dynamic tail as a cache anchor.
Can you improve this documentation? These fine people already did:
DeadMeme5441 & DeadMemeEdit on GitHub
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |