vaelii.impl.llm.ollama — com.vaelii/vaelii 0.1.0

A local backend: Ollama's chat API over raw HTTP.

Same shape as vaelii.impl.llm.anthropic — the neutral request/response maps of vaelii.impl.llm.protocol, an explicit connection handle, no global state, no new dependency (cheshire for JSON, JDK java.net.http). What differs is everything the transport does:

• No credential. Ollama serves on a host, not behind an API key, so available? is a reachability probe rather than a credential lookup. That is what makes this backend testable end to end.
• The context window is the caller's to set — options.num_ctx, per request. Ollama silently truncates a prompt longer than it, which would quietly drop the user's selection, so the caller sizes the prompt against num-ctx before sending (vaelii.impl.llm.selection/budget-problem); nothing here truncates.
• Constrained decoding instead of tool calls. :format carries a JSON schema that the sampler is restricted to, so a model with no tools capability still answers in an exact shape. capabilities reads what a model can actually do, and supports-tools? is the gate — sending 42 tool schemas to a completion-only model spends the whole window on something it will never emit.
• Streaming is newline-delimited JSON, not SSE: one object per token-ish chunk, the last carrying done: true and the run's counts.

Counts come back on every response — prompt_eval_count is the measured prompt size, which is what a budget is checked against after the fact.

available? clj

(available?)

(available? opts)

Is an Ollama reachable? The gate a caller uses to decide between this backend and the stub, and the one the live tests skip on.

base-url clj

(base-url)

The Ollama base URL: VAELII_OLLAMA_HOST, else OLLAMA_HOST when it names a real target, else default-host.

body clj

(body {:keys [model system messages tools num-ctx max-tokens temperature
              keep-alive]
       fmt :format}
      {:keys [stream?]})

The neutral request -> Ollama's /api/chat body.

:format (a JSON schema) constrains decoding, which is how a model with no tools capability is held to an exact output shape. :num-ctx sizes the window, :tools is sent only when non-empty, and temperature defaults to 0 so a proposal is reproducible.

keep_alive is always sent (configured-keep-alive, 30 minutes by default), because model load is the whole latency of a local turn and a host left to its own 5-minute default evicts the model between one reader's question and the next.

capabilities clj

(capabilities model)

(capabilities model opts)

The capability keywords a model declares — #{:completion}, #{:completion :tools}, … — or nil when it cannot be read. A model without :tools cannot tool-call: its chat template has no tools section, so schemas sent to it are window spent on nothing.

configured-generation-model clj

(configured-generation-model)

The model the page-generation path runs: VAELII_OLLAMA_GENERATION_MODEL when set, else default-generation-model.

configured-keep-alive clj

(configured-keep-alive)

How long to ask the host to hold the model resident: VAELII_OLLAMA_KEEP_ALIVE when set, else default-keep-alive.

configured-model clj

(configured-model)

The model to run: VAELII_OLLAMA_MODEL when set, else default-model.

configured-num-ctx clj

(configured-num-ctx)

The context window to ask for: VAELII_OLLAMA_NUM_CTX when set and numeric, else default-num-ctx.

context-length clj

(context-length model)

(context-length model opts)

The model's native context window from /api/show's model_info, or nil. The ceiling a caller's num-ctx must stay under; default-num-ctx sits well below it.

default-generation-model clj

The model the page-generation path runs (vaelii.impl.llm.session/propose-page) unless told otherwise. VAELII_OLLAMA_GENERATION_MODEL overrides.

Measured on the flesh-out task — write new type-level common sense about a term, under page/output-schema:

model	admissible assertions	warm wall clock	notes
qwen3-coder:30b	20/20	1.9 s	the default; needs the vocabulary card to stop coining
phi4:14b	0	—	nothing usable under either output contract

Generation is a different job from rewriting shown lines, so the two paths default differently rather than sharing one compromise.

default-host clj

The Ollama this backend talks to unless told otherwise — Ollama's own default address, on the machine running this code. A host on the network is somebody's deployment and not a default: VAELII_OLLAMA_HOST names one, and OLLAMA_HOST is read too, so pointing this at another box is configuration rather than an edit.

default-keep-alive clj

How long the host holds the model resident after a turn. VAELII_OLLAMA_KEEP_ALIVE overrides, and a request's :keep-alive overrides per call.

Deliberately long, because the latency of a local turn is model load, not generation: measured on the page-generation task, three identical calls took 11.33 s, then 0.39 s, then 0.30 s — the first paid for reading 18 GB of weights off disk and the rest paid nothing. Prefill of a full page prompt is 0.02–0.20 s and generation 1.2–2.5 s, so a resident model is the whole difference between a usable panel and an unusable one.

default-model clj

The model this backend runs unless told otherwise. VAELII_OLLAMA_MODEL overrides, and provider's :model overrides per call.

Measured on the selection-editing task, for a caller trading latency against rigor:

model	on this task	notes
phi4:14b	1.7 s	the default; strong at rewriting shown lines, weak at coining new ones
qwen3.6:27b	11.8 s	better judgement; ignores format, answers in the line format anyway
qwen2.5-coder:32b	20.2 s	the most formally reliable; honours format

Editing and generating want different models. This one is for the edit path, where it is both correct and the fastest thing available; it produces nothing usable on the page-generation path under either output contract. That path's model is default-generation-model.

default-num-ctx clj

The context window a request asks for unless told otherwise. Deliberately far under the model's native ceiling: a selection-scoped prompt is small, and a smaller window is less KV cache, a faster prefill, and a hard budget that surfaces an oversized selection as a refusal instead of a silent truncation. VAELII_OLLAMA_NUM_CTX overrides.

default-num-predict clj

The output cap. A rewritten selection is roughly one JSON object per selected sentex, so this bounds a runaway generation without cutting an honest answer short.

generation-provider clj

(generation-provider)

(generation-provider opts)

An Ollama provider on the page-generation model (configured-generation-model), for vaelii.impl.llm.session/propose-page. opts is provider's, and :model there still wins.

It exists to name the pairing: editing and generating are different jobs with different best models, and a caller that reaches for provider on a page turn gets the edit model, which produces nothing usable there.

normalize-host clj

(normalize-host host)

A host string -> the base URL to call. A host is conventionally written bare (localhost:11434, 127.0.0.1), so a missing scheme is filled in and a trailing slash dropped.

parse-response clj

(parse-response m)

Ollama's chat JSON -> the neutral response map (vaelii.impl.llm.protocol).

provider clj

(provider)

(provider {:keys [host model num-ctx keep-alive timeout-ms]
           :or {timeout-ms 300000}})

An Ollama-backed Provider. opts:

:host default OLLAMA_HOST, else default-host :model default VAELII_OLLAMA_MODEL, else default-model :num-ctx default VAELII_OLLAMA_NUM_CTX, else default-num-ctx :keep-alive how long the host holds the model resident (default configured-keep-alive, 30 minutes) :timeout-ms default 300000 — a cold model load on a 14B is tens of seconds

:model / :num-ctx here are defaults a request may override. Nothing is validated at construction: building a provider opens no socket, so a caller that must know the host is up asks available?.

show clj

(show model)

(show model {:keys [host timeout-ms] :or {timeout-ms 10000}})

/api/show for one model — its template, parameters, and declared capabilities. nil when the host or the model is not reachable.

supports-tools? clj

(supports-tools? model)

(supports-tools? model opts)

Can this model use tool schemas? False for a completion-only model, and false when the host cannot be reached — the conservative answer, since the cost of guessing yes is a wasted context window.

version clj

(version)

(version {:keys [host timeout-ms] :or {timeout-ms 2000}})

The Ollama server's version string, or nil when it is not reachable. timeout-ms is short by default: this is a gate, and a caller that must decide between this backend and the stub should not wait on a dead host.

warm clj

(warm)

(warm model)

(warm model
      {:keys [host timeout-ms keep-alive num-ctx prefill-tokens]
       :or {timeout-ms 300000 prefill-tokens 2048}})

Make a model ready to answer fast, and hold it that way. Returns {:loaded? bool :elapsed-ms n :model m :prefill-tokens n}.

Loading the weights is only half the cost. Measured on a 30B, with the model already resident, the first page turn still took 6.4 s to its first assertion while every turn after it took 0.40 s — the first real prefill pays for the window's KV cache and the compute path at that size. So this warms with a prefill of a realistic size (:prefill-tokens, 2048) at the same window a request will use, and generates one token. That moves the one-off 6 s into the warm-up: the turn after it is 0.7 s to first assertion.

Called when a page opens, it turns the reader's first question from a ten-second wait into a sub-second one. Bare, not warm! — it destroys nothing; it spends memory on the host.

opts: :host, :timeout-ms (300000), :keep-alive, :num-ctx, :prefill-tokens.