litellm.wrappers — tech.unravel/litellm-clj 0.3.0-alpha

Utility wrappers for provider calls with fallback, retry, timeout, and cost-tracking

compose-wrappers clj

(compose-wrappers wrappers config-name request-map completion-fn)

Compose multiple wrappers together.

Wrappers are applied right-to-left (like function composition).

Example: (compose-wrappers [(partial with-retry _ _ _ {:max-attempts 3}) (partial with-timeout _ _ _ {:timeout-ms 30000}) (partial with-cost-tracking _ _ _ log-cost)] :fast {:messages [...]} completion/chat)

with-cost-tracking clj

(with-cost-tracking config-name request-map completion-fn callback)

Track the cost of a completion call.

Args: config-name - Config keyword or map request-map - The request to send completion-fn - Function to call callback - Function to call with cost information: (fn [cost usage response] ...)

The callback receives:

• cost: Estimated cost in USD
• usage: Token usage map {:prompt-tokens N :completion-tokens N :total-tokens N}
• response: The full response

Example: (with-cost-tracking :fast {:messages [...]} completion/chat (fn [cost usage resp] (log/info "Request cost:" cost "USD")))

with-fallback clj

(with-fallback config-names request-map completion-fn)

Try multiple configurations in order until one succeeds.

Args: config-names - Vector of config keywords to try in order request-map - The request to send completion-fn - Function to call with resolved config and request

Example: (with-fallback [:fast :cheap-fallback] {:messages [...]} completion/chat)

with-retry clj

(with-retry config-name request-map completion-fn opts)

Retry a completion call with exponential backoff.

Args: config-name - Config keyword or map request-map - The request to send completion-fn - Function to call opts - Options map with: :max-attempts (default 3) :backoff-ms (default 1000) :max-backoff-ms (default 30000) :retry-on - Optional predicate to determine if error should be retried

Example: (with-retry :fast {:messages [...]} completion/chat {:max-attempts 3 :backoff-ms 1000})

with-timeout clj

(with-timeout config-name request-map completion-fn opts)

Execute a completion call with a timeout.

Args: config-name - Config keyword or map request-map - The request to send completion-fn - Function to call opts - Options map with: :timeout-ms (required) - Timeout in milliseconds

Example: (with-timeout :fast {:messages [...]} completion/chat {:timeout-ms 30000})