Liking cljdoc? Tell your friends :D

com.blockether.svar.internal.codes

Fenced code-block extraction from raw LLM text responses.

Pure parsing. No HTTP, no provider knowledge. Used by ask-code! to turn a plain-text completion into a vector of tagged code blocks the caller reads/evals directly.

Extraction rules — extract-code-blocks recognizes three shapes:

  1. Tagged fence: clojure\n…\n → {:lang "clojure" :source …}
  2. Untagged fence: \n…\n → {:lang nil :source …}
  3. No fence at all: entire response → {:lang nil :source …}

select-blocks then enforces strict lang matching: ONLY blocks whose :lang equals the caller-supplied target survive. Untagged blocks (:lang nil) — including the fenceless-fallback case — are DROPPED. Models that want their code accepted MUST tag their fence with the requested lang.

Fenced code-block extraction from raw LLM text responses.

Pure parsing. No HTTP, no provider knowledge. Used by `ask-code!` to turn
a plain-text completion into a vector of tagged code blocks the caller
reads/evals directly.

Extraction rules — `extract-code-blocks` recognizes three shapes:
  1. Tagged fence:        ```clojure\n…\n```   →  {:lang "clojure" :source …}
  2. Untagged fence:      ```\n…\n```          →  {:lang nil       :source …}
  3. No fence at all:     entire response          →  {:lang nil       :source …}

`select-blocks` then enforces strict lang matching: ONLY blocks whose
`:lang` equals the caller-supplied target survive. Untagged blocks
(`:lang nil`) — including the fenceless-fallback case — are DROPPED.
Models that want their code accepted MUST tag their fence with the
requested lang.
raw docstring

com.blockether.svar.internal.guard

Input guardrails for LLM interactions.

Provides factory functions that create guards to validate user input:

  • static - Pattern-based detection of prompt injection attempts
  • moderation - LLM-based content policy violation detection (requires :ask-fn)
  • guard - Runs one or more guards on input

Guards are functions that take input and return it unchanged on success, or throw ExceptionInfo on violation.

Usage: (require '[com.blockether.svar.core :as svar]) (def my-guards [(static) (moderation {:ask-fn svar/ask! :policies #{:hate}})]) (-> user-input (guard my-guards) (svar/ask! ...))

Input guardrails for LLM interactions.

Provides factory functions that create guards to validate user input:
- `static` - Pattern-based detection of prompt injection attempts
- `moderation` - LLM-based content policy violation detection (requires :ask-fn)
- `guard` - Runs one or more guards on input

Guards are functions that take input and return it unchanged on success,
or throw ExceptionInfo on violation.

Usage:
(require '[com.blockether.svar.core :as svar])
(def my-guards [(static) 
                (moderation {:ask-fn svar/ask! :policies #{:hate}})])
(-> user-input
    (guard my-guards)
    (svar/ask! ...))
raw docstring

com.blockether.svar.internal.humanize

AI response humanization module.

Removes AI-style phrases and patterns from LLM outputs to make responses sound more natural and human-like.

Two tiers of patterns:

  • SAFE_PATTERNS (default): AI identity, refusal, knowledge, punctuation. Unambiguously AI-generated; safe for arbitrary text.
  • AGGRESSIVE_PATTERNS (opt-in): hedging, overused verbs/adjectives/nouns, opening/closing cliches. May match valid English in non-AI text.
AI response humanization module.

Removes AI-style phrases and patterns from LLM outputs to make responses
sound more natural and human-like.

Two tiers of patterns:
- SAFE_PATTERNS (default): AI identity, refusal, knowledge, punctuation.
  Unambiguously AI-generated; safe for arbitrary text.
- AGGRESSIVE_PATTERNS (opt-in): hedging, overused verbs/adjectives/nouns,
  opening/closing cliches. May match valid English in non-AI text.
raw docstring

com.blockether.svar.internal.jsonish

Wrapper for the JsonishParser Java class.

Provides SAP (Schemaless Adaptive Parsing) for malformed JSON from LLMs. Handles unquoted keys/values, trailing commas, markdown code blocks, etc.

Wrapper for the JsonishParser Java class.

Provides SAP (Schemaless Adaptive Parsing) for malformed JSON from LLMs.
Handles unquoted keys/values, trailing commas, markdown code blocks, etc.
raw docstring

com.blockether.svar.internal.llm

LLM client layer: HTTP transport, message construction, and all LLM interaction functions (ask!, abstract!, eval!, refine!, models!, sample!).

LLM client layer: HTTP transport, message construction, and all LLM interaction
functions (ask!, abstract!, eval!, refine!, models!, sample!).
raw docstring

com.blockether.svar.internal.modelsdev

models.dev catalog loader.

Reads the bundled resources/models.dev.json snapshot (refreshed via make refresh-models) and exposes a normalized view that downstream router code merges with KNOWN_PROVIDERS wire/policy overlay.

Catalog wins for: pricing, context, modalities, capability flags, release dates, family. svar overlay wins for: api-style, reasoning-style, llm-headers, env-keys, base-url, paths, extra-body, exclude-models, rate budgets, default-models.

Plan-vs-retail pricing — per-provider entries on models.dev already reflect plan zeros (e.g. github-copilot, zai-coding-plan ship {input:0, output:0}). For svar's :openai-codex and :anthropic-coding-plan we explicitly want retail pricing (the user pays at API rates once metered), so the overlay declares :pricing-source to redirect catalog lookup to the retail provider.

models.dev catalog loader.

Reads the bundled `resources/models.dev.json` snapshot (refreshed via
`make refresh-models`) and exposes a normalized view that downstream
router code merges with `KNOWN_PROVIDERS` wire/policy overlay.

Catalog wins for: pricing, context, modalities, capability flags,
release dates, family.
svar overlay wins for: api-style, reasoning-style, llm-headers,
env-keys, base-url, paths, extra-body, exclude-models, rate budgets,
default-models.

Plan-vs-retail pricing — per-provider entries on models.dev already
reflect plan zeros (e.g. `github-copilot`, `zai-coding-plan` ship
{input:0, output:0}). For svar's `:openai-codex` and
`:anthropic-coding-plan` we explicitly want **retail** pricing
(the user pays at API rates once metered), so the overlay declares
`:pricing-source` to redirect catalog lookup to the retail provider.
raw docstring

com.blockether.svar.internal.spec

Structured output specification system for LLM responses.

This namespace provides a DSL for defining expected output structures, converting specs to LLM prompts, and parsing LLM responses back to Clojure data.

Primary functions:

  • field - Define a field with name, type, cardinality, and description
  • spec - Create a spec from field definitions
  • build-ref-registry - Build a registry of referenced specs for nested types
  • spec->prompt - Generate LLM prompt text from a spec (sent to LLM)
  • str->data - Parse LLM response string to Clojure data (schemaless)
  • str->data-with-spec - Parse LLM response with spec-based type coercion
  • validate-data - Validate parsed data against a spec
  • data->str - Serialize Clojure data to JSON string

Data Flow:

  1. Define spec with spec and field functions
  2. Generate prompt with spec->prompt (sent to LLM)
  3. Parse response with str->data-with-spec (LLM response -> typed Clojure map)
  4. Optionally validate with validate-data
  5. Optionally serialize with data->str
Structured output specification system for LLM responses.

This namespace provides a DSL for defining expected output structures,
converting specs to LLM prompts, and parsing LLM responses back to Clojure data.

Primary functions:
- `field` - Define a field with name, type, cardinality, and description
- `spec` - Create a spec from field definitions
- `build-ref-registry` - Build a registry of referenced specs for nested types
- `spec->prompt` - Generate LLM prompt text from a spec (sent to LLM)
- `str->data` - Parse LLM response string to Clojure data (schemaless)
- `str->data-with-spec` - Parse LLM response with spec-based type coercion
- `validate-data` - Validate parsed data against a spec
- `data->str` - Serialize Clojure data to JSON string

Data Flow:
1. Define spec with `spec` and `field` functions
2. Generate prompt with `spec->prompt` (sent to LLM)
3. Parse response with `str->data-with-spec` (LLM response -> typed Clojure map)
4. Optionally validate with `validate-data`
5. Optionally serialize with `data->str`
raw docstring

com.blockether.svar.internal.usage

Canonical token-usage shape — single source of truth across providers.

Phase A of svar 0.6.0. Replaces the hybrid pre-0.6 shape that emitted :prompt_tokens with provider-dependent semantics (Anthropic additive, OpenAI inclusive) under the same key. Every provider normalizer now produces the SAME shape; downstream consumers read one set of keys regardless of which model served the call.

Canonical shape — INVARIANT: regular + cache-write + cache-read = input-tokens:

{:input-tokens <long> ;; TOTAL prompt tokens (always inclusive) :output-tokens <long> ;; TOTAL completion tokens :input-tokens-details {:regular <long> ;; not from cache, not written :cache-write <long> ;; written this request (1.25× input rate, anthropic; 0 else) :cache-read <long>} ;; served from cache (0.1× input rate, anthropic; ~10-50% off, openai) :output-tokens-details {:reasoning <long>} ;; subset of output-tokens :total-tokens <long> ;; convenience = input-tokens + output-tokens :raw <map>} ;; original provider envelope (debug / forensics)

Provider differences:

  • Anthropic Messages API (:anthropic api-style): RAW input_tokens excludes cached AND cache-creation. Canonical :input-tokens adds all three so the value is TOTAL.

  • OpenAI Chat / Responses (:openai-compatible-* api-styles): RAW prompt_tokens / input_tokens IS the total. Cached subset lives under prompt_tokens_details.cached_tokens / input_tokens_details.cached_tokens. No native cache-write concept (server-managed implicit caching), so :cache-write is always 0 here UNLESS the provider proxies Anthropic via OpenRouter and surfaces cache_creation_input_tokens as a pydantic extra field.

  • Z.ai (GLM coding-plan / OpenAI-compatible-chat): same as OpenAI.

Industry alignment (May 2026):

  • Vercel AI SDK V3 spec (vercel/ai#9921): inputTokens always TOTAL with inputTokensDetails {regular, cacheWrite, cacheRead}.
  • OpenTelemetry gen_ai.usage.input_tokens (≥ v1.37): SHOULD be inclusive (all kinds of input tokens).
  • Claude Code official statusline JSON: context_window.total_input_tokens = input + cache_creation + cache_read.

Reject: the additive convention (e.g. litellm PR #23342) leaves total_tokens inconsistent with prompt_tokens and breaks naive aggregation downstream.

Canonical token-usage shape — single source of truth across providers.

Phase A of svar 0.6.0. Replaces the hybrid pre-0.6 shape that emitted
`:prompt_tokens` with provider-dependent semantics (Anthropic
additive, OpenAI inclusive) under the same key. Every provider
normalizer now produces the SAME shape; downstream consumers read
one set of keys regardless of which model served the call.

Canonical shape — INVARIANT: `regular + cache-write + cache-read = input-tokens`:

  {:input-tokens          <long>   ;; TOTAL prompt tokens (always inclusive)
   :output-tokens         <long>   ;; TOTAL completion tokens
   :input-tokens-details  {:regular     <long>   ;; not from cache, not written
                           :cache-write <long>   ;; written this request (1.25× input rate, anthropic; 0 else)
                           :cache-read  <long>}  ;; served from cache (0.1× input rate, anthropic; ~10-50% off, openai)
   :output-tokens-details {:reasoning   <long>}  ;; subset of output-tokens
   :total-tokens          <long>   ;; convenience = input-tokens + output-tokens
   :raw                   <map>}   ;; original provider envelope (debug / forensics)

Provider differences:

- Anthropic Messages API (`:anthropic` api-style): RAW
  `input_tokens` excludes cached AND cache-creation. Canonical
  `:input-tokens` adds all three so the value is TOTAL.

- OpenAI Chat / Responses (`:openai-compatible-*` api-styles): RAW
  `prompt_tokens` / `input_tokens` IS the total. Cached subset lives
  under `prompt_tokens_details.cached_tokens` /
  `input_tokens_details.cached_tokens`. No native `cache-write`
  concept (server-managed implicit caching), so `:cache-write` is
  always 0 here UNLESS the provider proxies Anthropic via OpenRouter
  and surfaces `cache_creation_input_tokens` as a pydantic extra
  field.

- Z.ai (GLM coding-plan / OpenAI-compatible-chat): same as OpenAI.

Industry alignment (May 2026):

- Vercel AI SDK V3 spec (vercel/ai#9921): `inputTokens` always TOTAL
  with `inputTokensDetails {regular, cacheWrite, cacheRead}`.
- OpenTelemetry `gen_ai.usage.input_tokens` (≥ v1.37): SHOULD be
  inclusive (all kinds of input tokens).
- Claude Code official statusline JSON:
  `context_window.total_input_tokens = input + cache_creation + cache_read`.

Reject: the additive convention (e.g. litellm PR #23342) leaves
`total_tokens` inconsistent with `prompt_tokens` and breaks naive
aggregation downstream.
raw docstring

com.blockether.svar.internal.util

Shared internal utilities.

Shared internal utilities.
raw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close