Single source of truth for provider/gateway failure classification.
Why this namespace exists: llm (low-level HTTP retry) and router
(provider fallback) each grew their own copy of the same substring/status
heuristics, so a newly observed transient had to be added twice and the two
layers could disagree about whether the very same exception was retryable.
Everything about "what kind of failure is this and is it safe to retry"
now lives here, and both layers delegate.
The driving deployment is a self-hosted LiteLLM gateway in front of many upstreams (Bedrock, Azure, vLLM). Under load it produces a steady drizzle of failures that are NOT the caller's fault and NOT the model's answer:
litellm.Timeout: BedrockException: Timeout Error - litellm.Timeout: Connection timed out. Timeout passed=Timeout(connect=5.0, read=600.0 ...) Received Model Group=claude-sonnet-5 Available Model Group Fallbacks=None
HTTP/1.1 header parser received no bytes
No deployments available for selected model group
Every one of those clears on a retry seconds later. Agents that feel "bulletproof" against such a gateway are not doing anything clever: they simply classify the whole transient family as retryable and back off with jitter, instead of surfacing the first blip as a hard failure.
Classification is deliberately EVIDENCE-ORDERED, most specific first: account state (quota/billing) and definitive client errors win over any transient-looking wording, so a 400 whose body merely contains the word "timeout" is never retried.
All predicates are pure: they take an already-lower-cased haystack plus an optional status, or a Throwable, and return data. No IO, no sleeping.
Single source of truth for provider/gateway failure classification. Why this namespace exists: `llm` (low-level HTTP retry) and `router` (provider fallback) each grew their own copy of the same substring/status heuristics, so a newly observed transient had to be added twice and the two layers could disagree about whether the very same exception was retryable. Everything about "what kind of failure is this and is it safe to retry" now lives here, and both layers delegate. The driving deployment is a self-hosted LiteLLM gateway in front of many upstreams (Bedrock, Azure, vLLM). Under load it produces a steady drizzle of failures that are NOT the caller's fault and NOT the model's answer: litellm.Timeout: BedrockException: Timeout Error - litellm.Timeout: Connection timed out. Timeout passed=Timeout(connect=5.0, read=600.0 ...) Received Model Group=claude-sonnet-5 Available Model Group Fallbacks=None HTTP/1.1 header parser received no bytes No deployments available for selected model group Every one of those clears on a retry seconds later. Agents that feel "bulletproof" against such a gateway are not doing anything clever: they simply classify the whole transient family as retryable and back off with jitter, instead of surfacing the first blip as a hard failure. Classification is deliberately EVIDENCE-ORDERED, most specific first: account state (quota/billing) and definitive client errors win over any transient-looking wording, so a 400 whose body merely contains the word "timeout" is never retried. All predicates are pure: they take an already-lower-cased haystack plus an optional status, or a Throwable, and return data. No IO, no sleeping.
(backoff-ms base max-ms)(backoff-ms base max-ms rand-fn)Exponential backoff with FULL JITTER, the AWS-recommended shape.
base is the un-jittered delay for this attempt (already multiplied), and
the returned sleep is uniform in [base/2, base]. Jitter matters precisely in
the failure mode this namespace targets: when one gateway hiccups, every
in-flight agent retries at the same instant and re-creates the overload.
rand-fn is injectable so tests stay deterministic.
Exponential backoff with FULL JITTER, the AWS-recommended shape. `base` is the un-jittered delay for this attempt (already multiplied), and the returned sleep is uniform in [base/2, base]. Jitter matters precisely in the failure mode this namespace targets: when one gateway hiccups, every in-flight agent retries at the same instant and re-creates the overload. `rand-fn` is injectable so tests stay deterministic.
Stable, user-facing failure families. :retryable? is svar's own retry
verdict; :reached-model? answers the question a user actually asks after a
failure — did anything run, and can I safely send this again?
Stable, user-facing failure families. `:retryable?` is svar's own retry verdict; `:reached-model?` answers the question a user actually asks after a failure — did anything run, and can I safely send this again?
(classify e)Classifies a provider/gateway failure into a stable, actionable shape.
Returns:
{:category one of CATEGORIES
:retryable? safe for svar to re-send automatically
:reached-model? true / false / nil (unknown)
:status upstream HTTP status, when any
:request-id correlation id, when the gateway printed one
:summary one-line human explanation
:next-step one-line human guidance}
Order matters: account state and definitive client errors are decided before any transient wording, so a 400 that merely mentions "timeout" stays a hard failure.
Classifies a provider/gateway failure into a stable, actionable shape.
Returns:
{:category one of `CATEGORIES`
:retryable? safe for svar to re-send automatically
:reached-model? true / false / nil (unknown)
:status upstream HTTP status, when any
:request-id correlation id, when the gateway printed one
:summary one-line human explanation
:next-step one-line human guidance}
Order matters: account state and definitive client errors are decided before
any transient wording, so a 400 that merely mentions "timeout" stays a hard
failure.How long ONE successful connection keeps a host classified as 'healthy'.
A message-less java.net.ConnectException (JDK 25 collapses ECONNREFUSED,
host-down and transient blips into one indistinguishable, message-free
shape) is retried ONLY when THIS host connected successfully inside this
window — recent proof the network path works, so the failure is a transient
blip. A host never reached (LM Studio/Ollama down, wrong port) is absent
from the registry and fails fast instead of eating retry backoff.
How long ONE successful connection keeps a host classified as 'healthy'. A message-less `java.net.ConnectException` (JDK 25 collapses ECONNREFUSED, host-down and transient blips into one indistinguishable, message-free shape) is retried ONLY when THIS host connected successfully inside this window — recent proof the network path works, so the failure is a transient blip. A host never reached (LM Studio/Ollama down, wrong port) is absent from the registry and fails fast instead of eating retry backoff.
(connection-error->ex-info e url)Wrap a connect-phase failure in an ex-info carrying a human-readable,
provider-aware message plus the :url that could not be reached — the raw
java.net.ConnectException message is often nil/terse with no hint of which
provider/endpoint died. Keeps the original throwable as cause so
transport-retryable? still walks the real java.net.* exception.
Wrap a connect-phase failure in an `ex-info` carrying a human-readable, provider-aware message plus the `:url` that could not be reached — the raw `java.net.ConnectException` message is often nil/terse with no hint of which provider/endpoint died. Keeps the original throwable as cause so `transport-retryable?` still walks the real `java.net.*` exception.
(connection-error-reason e)Human phrase for a connect-phase failure. Prefers a real (non-blank) message
from the cause chain, but on JDK 25 / java.net.http the ConnectException
chain is frequently all-nil messages, so fall back to a class-derived phrase
instead of leaking a raw classname to the user.
Human phrase for a connect-phase failure. Prefers a real (non-blank) message from the cause chain, but on JDK 25 / `java.net.http` the `ConnectException` chain is frequently all-nil messages, so fall back to a class-derived phrase instead of leaking a raw classname to the user.
(connection-error? e)True when any link in the cause chain is a connect-phase network failure:
the TCP/TLS connection to the provider could not be established at all
(refused, host down/unreachable, DNS miss, connect-phase timeout). Distinct
from mid-stream transport drops, which already carry an HTTP envelope and
:stream? data and are classified by transport-retryable?.
True when any link in the cause chain is a connect-phase network failure: the TCP/TLS connection to the provider could not be established at all (refused, host down/unreachable, DNS miss, connect-phase timeout). Distinct from mid-stream transport drops, which already carry an HTTP envelope and `:stream?` data and are classified by `transport-retryable?`.
svar's OWN watchdog/caller stream aborts. Each is a DELIBERATE InputStream
.close (idle/semantic watchdog fired, or caller cancel) — NOT a transient
peer connection drop. The JDK surfaces the intentional close as an
IOException whose message is 'Stream closed', which then trips the broad
'closed' substring heuristic. Left unguarded, the HTTP retry loop misreads a
watchdog abort as a retryable connection blip and silently re-hammers the
SAME provider. These belong to bounded, observable router-level fallback.
svar's OWN watchdog/caller stream aborts. Each is a DELIBERATE `InputStream` `.close` (idle/semantic watchdog fired, or caller cancel) — NOT a transient peer connection drop. The JDK surfaces the intentional close as an `IOException` whose message is 'Stream closed', which then trips the broad 'closed' substring heuristic. Left unguarded, the HTTP retry loop misreads a watchdog abort as a retryable connection blip and silently re-hammers the SAME provider. These belong to bounded, observable router-level fallback.
(ex-chain e)Exception + its causes, capped so a self-referential chain cannot loop.
Exception + its causes, capped so a self-referential chain cannot loop.
(haystack e)Lower-cased message + body of an exception, the text every predicate here
consumes. Body first: the provider's own words are more specific than the
HTTP wrapper's Exceptional status code: 400.
Lower-cased message + body of an exception, the text every predicate here consumes. Body first: the provider's own words are more specific than the HTTP wrapper's `Exceptional status code: 400`.
(healthy-host-connect-blip? e)True when a message-less ConnectException hit a host that connected successfully moments ago. The prior success is evidence the network is healthy, so this is a transient blip worth retrying; a host never reached stays non-retryable and fails fast.
True when a message-less ConnectException hit a host that connected successfully moments ago. The prior success is evidence the network is healthy, so this is a transient blip worth retrying; a host never reached stays non-retryable and fails fast.
host -> epoch-ms of its most recent successful connection. Drives the health
gate for message-less ConnectExceptions (see CONNECT_HEALTH_WINDOW_MS).
host -> epoch-ms of its most recent successful connection. Drives the health gate for message-less ConnectExceptions (see `CONNECT_HEALTH_WINDOW_MS`).
(host-connection-healthy? url)True when url's host connected successfully within
CONNECT_HEALTH_WINDOW_MS — recent proof the network path to it works.
True when `url`'s host connected successfully within `CONNECT_HEALTH_WINDOW_MS` — recent proof the network path to it works.
(mark-connection-healthy! url)Record that url's host just connected successfully (HTTP response/headers
received).
Record that `url`'s host just connected successfully (HTTP response/headers received).
(message-less-connect-exception? e)True when the cause chain holds a java.net.ConnectException with a blank
message — the JDK-25 shape that collapses ECONNREFUSED, host-down and
transient connectivity blips into one indistinguishable exception that
cannot be classified by message substring alone.
True when the cause chain holds a `java.net.ConnectException` with a blank message — the JDK-25 shape that collapses ECONNREFUSED, host-down and transient connectivity blips into one indistinguishable exception that cannot be classified by message substring alone.
(next-delay-ms e base max-ms)(next-delay-ms e base max-ms rand-fn)Sleep before the next attempt: a server-declared Retry-After wins (clamped
to max-ms), otherwise jittered exponential backoff.
Sleep before the next attempt: a server-declared `Retry-After` wins (clamped to `max-ms`), otherwise jittered exponential backoff.
Lower-cased substrings that mark provider subscription/quota/billing exhaustion — a hard account state, NOT a transient throttle. Even on HTTP 429 these must NOT retry (mirrors pi's NON_RETRYABLE_PROVIDER_LIMIT_ERROR pattern in packages/ai/src/utils/retry.ts).
Lower-cased substrings that mark provider subscription/quota/billing exhaustion — a hard account state, NOT a transient throttle. Even on HTTP 429 these must NOT retry (mirrors pi's NON_RETRYABLE_PROVIDER_LIMIT_ERROR pattern in packages/ai/src/utils/retry.ts).
(pre-response-drop? hay)True when the failure text proves the connection died before any response bytes arrived — the request never reached the model.
True when the failure text proves the connection died before any response bytes arrived — the request never reached the model.
(provider-limit-error? hay)True when error text/body names a subscription/quota/billing exhaustion (account state) that must never be retried as a transient throttle.
True when error text/body names a subscription/quota/billing exhaustion (account state) that must never be retried as a transient throttle.
(provider-limit-failure? status hay)The single hard-account-state question: does this status or body mean the account is out of quota/credit/budget? Never retryable, never re-routable.
The single hard-account-state question: does this status or body mean the account is out of quota/credit/budget? Never retryable, never re-routable.
HTTP statuses that ARE an account/billing wall on their own, whatever the body says. 402 Payment Required is the whole point of the code.
HTTP statuses that ARE an account/billing wall on their own, whatever the body says. 402 Payment Required is the whole point of the code.
(request-id e)Extracts a correlation id from an exception's headers or body, or nil.
Extracts a correlation id from an exception's headers or body, or nil.
(retry-after-ms e)Server-declared cooldown from a Retry-After header, in ms, or nil.
Only the delta-seconds form is honored; an HTTP-date form is ignored rather
than mis-parsed into a nonsense sleep.
Server-declared cooldown from a `Retry-After` header, in ms, or nil. Only the delta-seconds form is honored; an HTTP-date form is ignored rather than mis-parsed into a nonsense sleep.
(retryable? e)Convenience: svar's retry verdict for one exception.
Convenience: svar's retry verdict for one exception.
Regex over an error's already-lower-cased message+body marking a transient provider/transport failure that carries no mappable HTTP status.
Regex over an error's already-lower-cased message+body marking a transient provider/transport failure that carries no mappable HTTP status.
(stream-output-started? e)True when the provider already streamed visible content/reasoning before the failure. Such a call can NOT be silently retried — svar cannot rewind bytes the caller has already rendered.
True when the provider already streamed visible content/reasoning before the failure. Such a call can NOT be silently retried — svar cannot rewind bytes the caller has already rendered.
SSE ended before the provider's terminal marker (stream-truncated) or the
provider explicitly said response.incomplete (stream-incomplete). Both
are transport/provider failures rather than a model answer, and retry (as in
the OpenAI Codex CLI) usually succeeds — but only before visible output.
SSE ended before the provider's terminal marker (`stream-truncated`) or the provider explicitly said `response.incomplete` (`stream-incomplete`). Both are transport/provider failures rather than a model answer, and retry (as in the OpenAI Codex CLI) usually succeeds — but only before visible output.
Typed stream aborts safe to retry only before visible output. The low-level HTTP retry layer excludes these (it would wait one whole timeout per same-provider retry); the router instead performs observable provider fallback.
Typed stream aborts safe to retry only before visible output. The low-level HTTP retry layer excludes these (it would wait one whole timeout per same-provider retry); the router instead performs observable provider fallback.
(transient-error? e)(transient-error? e opts)The ROUTER's verdict: is this failure soft enough to try another provider /
another attempt? Broader than transport-retryable? because the router may
legitimately re-route a watchdog timeout or an HTTP status, but never a hard
account/quota wall and never after visible output has been streamed.
opts accepts :transient-status-codes (defaults to
TRANSIENT_STATUS_CODES).
The ROUTER's verdict: is this failure soft enough to try another provider / another attempt? Broader than `transport-retryable?` because the router may legitimately re-route a watchdog timeout or an HTTP status, but never a hard account/quota wall and never after visible output has been streamed. `opts` accepts `:transient-status-codes` (defaults to `TRANSIENT_STATUS_CODES`).
(transient-message-error? hay status)True when a statusless / wrapper / gateway transient shows up only in the
error TEXT. status gates out definitive client errors so a 400/404 whose
body happens to contain transient wording is never retried.
True when a statusless / wrapper / gateway transient shows up only in the error TEXT. `status` gates out definitive client errors so a 400/404 whose body happens to contain transient wording is never retried.
(transient-network-error? e)True when any link in the cause chain looks like a transient OS/network
connection error (see TRANSIENT_NETWORK_ERROR_SUBSTRINGS). Walks the whole
chain because babashka.http-client and the router wrap the raw java.net.*
exception several layers deep.
True when any link in the cause chain looks like a transient OS/network connection error (see `TRANSIENT_NETWORK_ERROR_SUBSTRINGS`). Walks the whole chain because babashka.http-client and the router wrap the raw `java.net.*` exception several layers deep.
Lower-cased substrings of transient OS/network-layer connection errors. A brief connectivity blip (wifi handoff, VPN reconnect, captive portal, laptop sleep/wake) surfaces these while the local stack is momentarily down — they clear once the network returns, so they are safe to retry with backoff instead of failing the whole call.
Deliberately excludes "connection refused" (ECONNREFUSED): a RST from the peer usually means a wrong endpoint / down service, not a transient blip, and retrying just hammers it.
Lower-cased substrings of transient OS/network-layer connection errors. A brief connectivity blip (wifi handoff, VPN reconnect, captive portal, laptop sleep/wake) surfaces these while the local stack is momentarily down — they clear once the network returns, so they are safe to retry with backoff instead of failing the whole call. Deliberately excludes "connection refused" (ECONNREFUSED): a RST from the peer usually means a wrong endpoint / down service, not a transient blip, and retrying just hammers it.
HTTP statuses that mean "try again", never "your request was wrong".
Beyond the classic 429/5xx set this includes:
408 — Request Timeout. LiteLLM maps litellm.Timeout (the Bedrock connect
timeout in the issue above) onto 408, and a bare 408 was previously
treated as a definitive 4xx client error and failed on the first blip.
425 — Too Early (TLS early-data replay refused by a fronting proxy).
520-527 — Cloudflare's origin-side family (unknown error, web server down,
connection timed out, origin unreachable, timeout occurred, SSL
handshake failed, invalid SSL certificate). Cloudflare fronts several
model providers, and 522/524 are pure transport blips.
598/599 — non-standard network read/connect timeout codes emitted by some
proxies (LiteLLM's httpx wrapper and nginx-based gateways).
Note 529 (Anthropic "overloaded") is standard here, and 429 is included but
provider-limit-error? vetoes it when the body proves account exhaustion.
HTTP statuses that mean "try again", never "your request was wrong".
Beyond the classic 429/5xx set this includes:
408 — Request Timeout. LiteLLM maps `litellm.Timeout` (the Bedrock connect
timeout in the issue above) onto 408, and a bare 408 was previously
treated as a definitive 4xx client error and failed on the first blip.
425 — Too Early (TLS early-data replay refused by a fronting proxy).
520-527 — Cloudflare's origin-side family (unknown error, web server down,
connection timed out, origin unreachable, timeout occurred, SSL
handshake failed, invalid SSL certificate). Cloudflare fronts several
model providers, and 522/524 are pure transport blips.
598/599 — non-standard network read/connect timeout codes emitted by some
proxies (LiteLLM's httpx wrapper and nginx-based gateways).
Note 529 (Anthropic "overloaded") is standard here, and 429 is included but
`provider-limit-error?` vetoes it when the body proves account exhaustion.(transport-retryable? e)SOFT transport failure: a truncated/reset/dropped connection with no HTTP status, safe for the low-level HTTP layer to retry against the SAME provider.
Excludes svar's own deliberate stream aborts (DELIBERATE_STREAM_ABORT_TYPES)
— their 'Stream closed' message is indistinguishable from a peer drop, and
retrying one silently re-hammers the provider for another full timeout.
SOFT transport failure: a truncated/reset/dropped connection with no HTTP status, safe for the low-level HTTP layer to retry against the SAME provider. Excludes svar's own deliberate stream aborts (`DELIBERATE_STREAM_ABORT_TYPES`) — their 'Stream closed' message is indistinguishable from a peer drop, and retrying one silently re-hammers the provider for another full timeout.
(url->host url)Host component of a URL string, or nil when absent/unparseable.
Host component of a URL string, or nil when absent/unparseable.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |