Sequential fallback across (provider, model) pairs.
Try each provider in order with the canonical request. Returns the first successful Response. If every attempt fails, throws ex-info carrying an :attempts vector of {:provider :model :error/...} maps in attempt order — callers can inspect to decide whether to re-raise, surface to UI, etc.
What this is NOT:
All of those are explicitly out of scope for the SDK. They are credential-pool plumbing, the exact kind of work LiteLLM's router.py exists for and that we delegate back to the calling application.
Reference: litellm-ref/router_utils/fallback_event_handlers.py:85 run_async_fallback (shape only — do NOT port the pool plumbing).
Sequential fallback across (provider, model) pairs.
Try each provider in order with the canonical request. Returns the
first successful Response. If every attempt fails, throws ex-info
carrying an :attempts vector of {:provider :model :error/...} maps
in attempt order — callers can inspect to decide whether to
re-raise, surface to UI, etc.
What this is NOT:
- Credential pools / multi-key load balancing
- Cooldown caches / weighted shuffle
- TPM/RPM enforcement / budget routing
- Latency-aware routing / complexity routing
All of those are explicitly out of scope for the SDK. They are
credential-pool plumbing, the exact kind of work LiteLLM's router.py
exists for and that we delegate back to the calling application.
Reference: litellm-ref/router_utils/fallback_event_handlers.py:85
run_async_fallback (shape only — do NOT port the pool plumbing).(with-fallbacks providers request)(with-fallbacks providers request {:keys [complete-fn on-attempt]})Try each [provider-id model-id] pair in sequence with the given canonical request. The request's :request/model is rewritten to the pair's model-id at each attempt.
Options: :complete-fn Custom completion fn (defaults to llm.sdk/complete, resolved lazily). Tests can pass a stub. :on-attempt Side-effect fn called with each attempt's failure map {:provider :model :error/reason ...} — handy for hooking up the application's logger without forcing a dependency in.
On all-fail, throws ex-info with: :attempts vector of failure maps in order :providers the original provider list
Try each [provider-id model-id] pair in sequence with the given
canonical request. The request's :request/model is rewritten to the
pair's model-id at each attempt.
Options:
:complete-fn Custom completion fn (defaults to llm.sdk/complete,
resolved lazily). Tests can pass a stub.
:on-attempt Side-effect fn called with each attempt's failure
map {:provider :model :error/reason ...} — handy
for hooking up the application's logger without
forcing a dependency in.
On all-fail, throws ex-info with:
:attempts vector of failure maps in order
:providers the original provider listcljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |