What every HTTP provider does that is not its wire format — the read deadline, the JSON-body refusal, and the excerpt both of them carry.
A provider namespace (vaelii.impl.llm.ollama, vaelii.impl.llm.anthropic) is two
things at once: an encoding of one vendor's messages, tools and stop reasons, and a
transport that posts bytes and reads a body under a deadline. The first is the whole
reason each exists separately and is not shared. The second was written out twice,
identically bar the vendor's name inside a string — and a deadline policy that lives in
two places is one that gets fixed in one.
Each provider passes an endpoint descriptor, {:label :slug}: :label is how the
far end is named in a message a reader sees ("the Ollama host", "the Anthropic
API"), :slug how it is named in a thread title. That pair is the entire difference
between the two copies this replaces.
What every HTTP provider does that is not its wire format — the read deadline, the
JSON-body refusal, and the excerpt both of them carry.
A provider namespace (`vaelii.impl.llm.ollama`, `vaelii.impl.llm.anthropic`) is two
things at once: an *encoding* of one vendor's messages, tools and stop reasons, and a
*transport* that posts bytes and reads a body under a deadline. The first is the whole
reason each exists separately and is not shared. The second was written out twice,
identically bar the vendor's name inside a string — and a deadline policy that lives in
two places is one that gets fixed in one.
Each provider passes an **endpoint** descriptor, `{:label :slug}`: `:label` is how the
far end is named in a message a reader sees ("the Ollama host", "the Anthropic
API"), `:slug` how it is named in a thread title. That pair is the entire difference
between the two copies this replaces.(decode {:keys [label]} text status)A JSON body -> data. Every refusal a provider raises carries a :type, and a 200
whose body is not JSON is one: a caller discriminates on the keyword, and the JSON
library's own exception is not something to discriminate on. A proxy in front of the
endpoint answering HTML is the case that reaches here. The text stays out of the
message and rides bounded in :excerpt.
A JSON body -> data. Every refusal a provider raises carries a `:type`, and a 200 whose body is not JSON is one: a caller discriminates on the keyword, and the JSON library's own exception is not something to discriminate on. A proxy in front of the endpoint answering HTML is the case that reaches here. The text stays out of the message and rides bounded in `:excerpt`.
(excerpt s)The opening of a body, for an exception's data. A response can be megabytes and what says why it would not parse is its first line, so the bound costs no diagnosis.
The opening of a body, for an exception's data. A response can be megabytes and what says *why* it would not parse is its first line, so the bound costs no diagnosis.
(under-read-deadline {:keys [label] :as endpoint} ms on-expiry f)Run f under a hard deadline on reading a response body: a daemon watchdog runs
on-expiry — closing the body, which is what makes a blocked read fail — once ms
have passed, and a failure raised after it fires is reported as :llm-timeout rather
than as whatever a closed socket happened to raise.
The request's own .timeout bounds the response arriving, and a streamed turn is
almost entirely what comes after that: the lines are pulled off the socket once send
has returned. This is what stops an endpoint that goes quiet mid-answer — a model
evicted under memory pressure is the way that happens locally — from holding the
calling thread for as long as it keeps the connection half-open. f consumes its
lines eagerly, so returning from it means the body is read.
Run `f` under a hard deadline on **reading** a response body: a daemon watchdog runs `on-expiry` — closing the body, which is what makes a blocked read fail — once `ms` have passed, and a failure raised after it fires is reported as `:llm-timeout` rather than as whatever a closed socket happened to raise. The request's own `.timeout` bounds the response *arriving*, and a streamed turn is almost entirely what comes after that: the lines are pulled off the socket once `send` has returned. This is what stops an endpoint that goes quiet mid-answer — a model evicted under memory pressure is the way that happens locally — from holding the calling thread for as long as it keeps the connection half-open. `f` consumes its lines eagerly, so returning from it means the body is read.
(watchdog {:keys [slug]} ms on-expiry)A daemon thread that runs on-expiry once ms have passed; interrupt it to cancel.
Daemon because a probe or a stalled read must never be the reason a JVM cannot exit.
A daemon thread that runs `on-expiry` once `ms` have passed; interrupt it to cancel. Daemon because a probe or a stalled read must never be the reason a JVM cannot exit.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |