Router-based configuration API for LiteLLM
Router-based configuration API for LiteLLM
(calculate-cost provider-name model prompt-tokens completion-tokens)Calculate cost for a request/response
Calculate cost for a request/response
(chat config-name message & {:keys [system-prompt] :as opts})Simple chat completion using registered configuration.
Examples: (chat :fast "What is 2+2?") (chat :smart "Explain quantum physics" :system-prompt "You are a physicist")
Simple chat completion using registered configuration. Examples: (chat :fast "What is 2+2?") (chat :smart "Explain quantum physics" :system-prompt "You are a physicist")
(clear-router!)Clear all registered configurations (useful for testing)
Clear all registered configurations (useful for testing)
(completion config-name request-map)Completion function using registered configurations.
First argument is a keyword referring to a registered config name.
Examples: ;; Using registered config (completion :fast {:messages [{:role :user :content "Hello"}]})
;; With additional options (completion :smart {:messages [{:role :user :content "Hello"}] :max-tokens 100})
Completion function using registered configurations.
First argument is a keyword referring to a registered config name.
Examples:
;; Using registered config
(completion :fast {:messages [{:role :user :content "Hello"}]})
;; With additional options
(completion :smart
{:messages [{:role :user :content "Hello"}]
:max-tokens 100})(create-router router-fn provider-configs)Helper to create a router configuration with multiple providers
Example: (create-router (fn [{:keys [priority]}] (if (= priority :high) {:provider :anthropic :model "claude-3-opus"} {:provider :openai :model "gpt-4o-mini"})) {:openai {:api-key "sk-..."} :anthropic {:api-key "sk-..."}})
Helper to create a router configuration with multiple providers
Example:
(create-router
(fn [{:keys [priority]}]
(if (= priority :high)
{:provider :anthropic :model "claude-3-opus"}
{:provider :openai :model "gpt-4o-mini"}))
{:openai {:api-key "sk-..."}
:anthropic {:api-key "sk-..."}})(estimate-request-tokens request)Estimate token count for a request
Estimate token count for a request
(estimate-tokens text)Estimate token count for text
Estimate token count for text
(extract-content response)Extract content from completion response
Extract content from completion response
(extract-message response)Extract message from completion response
Extract message from completion response
(extract-usage response)Extract usage information from response
Extract usage information from response
(get-config config-name)Get a registered configuration by name
Get a registered configuration by name
(list-configs)List all registered configuration names
List all registered configuration names
(list-providers)List all available providers
List all available providers
(provider-available? provider-name)Check if a provider is available
Check if a provider is available
(provider-info provider-name)Get information about a provider
Get information about a provider
(quick-setup!)Quick setup for common providers using environment variables
Sets up:
Quick setup for common providers using environment variables Sets up: - :openai if OPENAI_API_KEY is set - :anthropic if ANTHROPIC_API_KEY is set - :gemini if GEMINI_API_KEY is set - :mistral if MISTRAL_API_KEY is set - :ollama (always, defaults to localhost)
(register! config-name config-map)Register a provider configuration with a keyword name.
Configurations are stored in a registry and can be referenced by name in completion calls.
Configuration Types:
{:provider :openai :model "gpt-4" :config {...}}{:router router-fn :configs {...}}Examples:
;; Simple configuration
(register! :fast
{:provider :openai
:model "gpt-4o-mini"
:config {:api-key "sk-..."}})
;; Dynamic router configuration
(register! :adaptive
{:router (fn [{:keys [priority]}]
(if (= priority :high)
{:provider :anthropic :model "claude-3-opus-20240229"}
{:provider :openai :model "gpt-4o-mini"}))
:configs {:openai {:api-key "sk-..."}
:anthropic {:api-key "sk-ant-..."}}})
See also: completion, unregister!, list-configs
Register a provider configuration with a keyword name.
Configurations are stored in a registry and can be referenced by name in [[completion]] calls.
**Configuration Types:**
- **Simple:** `{:provider :openai :model "gpt-4" :config {...}}`
- **With router:** `{:router router-fn :configs {...}}`
**Examples:**
```clojure
;; Simple configuration
(register! :fast
{:provider :openai
:model "gpt-4o-mini"
:config {:api-key "sk-..."}})
;; Dynamic router configuration
(register! :adaptive
{:router (fn [{:keys [priority]}]
(if (= priority :high)
{:provider :anthropic :model "claude-3-opus-20240229"}
{:provider :openai :model "gpt-4o-mini"}))
:configs {:openai {:api-key "sk-..."}
:anthropic {:api-key "sk-ant-..."}}})
```
**See also:** [[completion]], [[unregister!]], [[list-configs]](setup-anthropic! &
{:keys [config-name api-key model]
:or {config-name :anthropic
model "claude-3-sonnet-20240229"}})Quick setup for Anthropic with optional custom config name
Quick setup for Anthropic with optional custom config name
(setup-gemini! &
{:keys [config-name api-key model]
:or {config-name :gemini model "gemini-pro"}})Quick setup for Gemini with optional custom config name
Quick setup for Gemini with optional custom config name
(setup-mistral! &
{:keys [config-name api-key model]
:or {config-name :mistral model "mistral-medium"}})Quick setup for Mistral with optional custom config name
Quick setup for Mistral with optional custom config name
(setup-ollama!
&
{:keys [config-name api-base model]
:or {config-name :ollama api-base "http://localhost:11434" model "llama3"}})Quick setup for Ollama with optional custom config name
Quick setup for Ollama with optional custom config name
(setup-openai! &
{:keys [config-name api-key model]
:or {config-name :openai model "gpt-4o-mini"}})Quick setup for OpenAI with optional custom config name
Quick setup for OpenAI with optional custom config name
(setup-openrouter! &
{:keys [config-name api-key model]
:or {config-name :openrouter model "openai/gpt-4"}})Quick setup for OpenRouter with optional custom config name
Quick setup for OpenRouter with optional custom config name
(supports-function-calling? provider-name)Check if provider supports function calling
Check if provider supports function calling
(supports-streaming? provider-name)Check if provider supports streaming
Check if provider supports streaming
(unregister! config-name)Remove a configuration from the router
Remove a configuration from the router
(validate-request provider-name request)Validate a request against provider capabilities
Validate a request against provider capabilities
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |