com.blockether.svar.core — com.blockether/svar 0.1.1

LLM interaction utilities for structured and unstructured outputs.

SVAR = Structured Validated Automated Reasoning

Provides main functions:

• ask! - Structured output using the spec DSL
• abstract! - Text summarization using Chain of Density prompting
• eval! - LLM self-evaluation for reliability and accuracy assessment
• refine! - Iterative refinement using decomposition and verification
• models! - Fetch available models from the LLM API
• sample! - Generate test data samples matching a spec

Guardrails:

• static-guard - Pattern-based prompt injection detection
• moderation-guard - LLM-based content moderation
• guard - Run one or more guards on input

Humanization:

• humanize-string - Strip AI-style phrases from text
• humanize-data - Humanize string values in data structures
• humanizer - Create a reusable humanizer function

PageIndex:

• index! - Index a document file (PDF, MD, TXT) and save structured data
• load-index - Load an indexed document from a pageindex directory

Re-exports spec DSL (field, spec, str->data, str->data-with-spec, data->str, validate-data, spec->prompt, build-ref-registry), RLM (create-env, register-env-fn!, register-env-def!, ingest-to-env!, dispose-env!, query-env!, pprint-trace, print-trace, generate-qa-env!), PageIndex (index!, load-index), and make-config so users can require only this namespace.

Configuration: Config MUST be passed explicitly to all LLM functions via the :config parameter. No global state. No dependency injection.

Example: (def config (make-config {:api-key "sk-..." :base-url "https://api.openai.com/v1"})) (ask! {:config config :spec my-spec :messages [(system "Help the user.") (user "What is 2+2?")] :model "gpt-4o"})

References:

• Chain of Density: https://arxiv.org/abs/2309.04269
• LLM Self-Evaluation: https://learnprompting.org/docs/reliability/lm_self_eval
• DuTy: https://learnprompting.org/docs/advanced/decomposition/duty-distinct-chain-of-thought
• CoVe: https://learnprompting.org/docs/advanced/self_criticism/chain_of_verification

abstract! clj

Creates a dense, entity-rich summary using Chain of Density prompting. See internal.llm for details.

ask! clj

Asks the LLM and returns structured Clojure data with token usage and cost. See internal.llm for details.

assistant clj

Creates an assistant message. See internal.llm for details.

build-ref-registry clj

Builds a registry of referenced specs. See spec namespace for details.

CARDINALITY clj

Field option: Field cardinality (:spec.cardinality/one or :spec.cardinality/many).

CARDINALITY_MANY clj

Cardinality: Vector of values.

CARDINALITY_ONE clj

Cardinality: Single value.

create-env clj

Creates an RLM environment for processing large contexts via iterative code execution. See internal.rlm for details.

data->str clj

Serializes Clojure data to LLM-compatible string. See spec namespace for details.

DESCRIPTION clj

Field option: Human-readable field description.

dispose-env! clj

Disposes an RLM environment and cleans up resources. See internal.rlm for details.

eval! clj

Evaluates an LLM output using LLM self-evaluation. See internal.llm for details.

field clj

Creates a field definition for a spec. See spec namespace for details.

generate-qa-env! clj

Generates question-answer pairs from ingested documents using a multi-stage pipeline. See internal.rlm for details.

guard clj

Runs guard(s) on input. See internal.guard for details.

GUARD_DEFAULT_INJECTION_PATTERNS clj

Default patterns for prompt injection detection.

GUARD_DEFAULT_MODERATION_POLICIES clj

Default OpenAI moderation policies to check.

HUMANIZE clj

Field option: When true, marks field for humanization via :humanizer in ask!.

humanize-data clj

Recursively humanizes all strings in a data structure. See internal.humanize for details.

humanize-string clj

Removes AI-style phrases from text to make it sound more natural. See internal.humanize for details.

HUMANIZE_AGGRESSIVE_PATTERNS clj

Aggressive humanization patterns: hedging, overused verbs/adjectives/nouns, cliches. May match valid English -- opt-in only via {:aggressive? true}.

HUMANIZE_DEFAULT_PATTERNS clj

Default patterns for AI phrase humanization (safe + aggressive combined). Preserved for backward compatibility.

HUMANIZE_SAFE_PATTERNS clj

Safe humanization patterns: AI identity, refusal, knowledge, punctuation. These are unambiguously AI artifacts, safe for arbitrary text.

humanizer clj

Creates a humanization function with optional custom patterns. See internal.humanize for details.

image clj

Creates an image attachment for use with user messages. See internal.llm for details.

index! clj

Index a document file (PDF, MD, TXT) and save structured data as EDN + PNG files.

Creates a .pageindex directory alongside the original file containing: document.edn — structured document data images/ — extracted images as PNG files

Opts: :config - LLM config override :pages - Page selector (1-indexed). Limits which pages are indexed. Supports: integer, [from to] range, or [[1 3] 5 [7 10]] mixed. nil = all pages (default).

Vision extraction: :vision-model - Model for vision page extraction :parallel - Max concurrent vision page extractions (default: 3)

Quality refinement: :refine? - Enable post-extraction quality refinement (default: false) :refine-model - Model for eval/refine steps (default: "gpt-4o") :parallel-refine - Max concurrent eval/refine operations (default: 2)

Example: (svar/index! "docs/manual.pdf") (svar/index! "docs/manual.pdf" {:pages [1 5]}) (svar/index! "docs/manual.pdf" {:pages [[1 3] 5 [7 10]]}) (svar/index! "docs/manual.pdf" {:vision-model "gpt-4o" :parallel 5 :refine? true :refine-model "gpt-4o-mini" :parallel-refine 3})

See internal.rlm.internal.pageindex.core for full options.

ingest-to-env! clj

Ingests documents into an RLM environment for querying. See internal.rlm for details.

KEY-NS clj

Spec option: Namespace prefix to add to keys during parsing.

load-index clj

Load an indexed document from a .pageindex directory.

Reads the EDN + PNG files produced by index! and returns the document map. Also supports loading legacy formats for backward compatibility.

Example: (svar/load-index "docs/manual.pageindex")

See internal.rlm.internal.pageindex.core for details.

make-config clj

Creates an LLM configuration map. See internal.config for details.

models! clj

Fetches available models from the LLM API. See internal.llm for details.

moderation-guard clj

Creates a guard function that uses LLM to check content against policies. See internal.guard for details.

NAME clj

Field option: Field name as Datomic-style keyword (e.g., :user/name).

pprint-trace clj

Pretty-prints an RLM trace to a string. See internal.rlm for details.

print-trace clj

Pretty-prints an RLM trace to stdout. See internal.rlm for details.

query-env! clj

Runs a query against an RLM environment using iterative code execution. See internal.rlm for details.

refine! clj

Iteratively refines LLM output using decomposition and verification. See internal.llm for details.

register-env-def! clj

Registers a constant in the RLM's SCI sandbox. See internal.rlm for details.

register-env-fn! clj

Registers a custom function in the RLM's SCI sandbox. See internal.rlm for details.

REQUIRED clj

Field option: Whether field is required (default: true). Set to false for optional.

sample! clj

Generates test data samples matching a spec with self-correction. See internal.llm for details.

save-qa! clj

Saves generate-qa-env! results to EDN and/or Markdown files. See internal.rlm for details.

spec clj

Creates a spec definition from field definitions. See spec namespace for details.

spec->prompt clj

Generates LLM prompt from a spec. See spec namespace for details.

static-guard clj

Creates a guard function that checks for prompt injection patterns. See internal.guard for details.

str->data clj

Parses LLM response string to Clojure data. See spec namespace for details.

str->data-with-spec clj

Parses LLM response with spec validation. See spec namespace for details.

system clj

Creates a system message. See internal.llm for details.

TARGET clj

Field option: Reference target for :spec.type/ref fields.

TYPE clj

Field option: Field type (e.g., :spec.type/string, :spec.type/int).

TYPE_BOOL clj

Type: Boolean value.

TYPE_DATE clj

Type: ISO date (YYYY-MM-DD).

TYPE_DATETIME clj

Type: ISO datetime.

TYPE_DOUBLE_V_1 clj

Type: Fixed-size double vector (1 element).

TYPE_DOUBLE_V_10 clj

Type: Fixed-size double vector (10 elements).

TYPE_DOUBLE_V_11 clj

Type: Fixed-size double vector (11 elements).

TYPE_DOUBLE_V_12 clj

Type: Fixed-size double vector (12 elements).

TYPE_DOUBLE_V_2 clj

Type: Fixed-size double vector (2 elements).

TYPE_DOUBLE_V_3 clj

Type: Fixed-size double vector (3 elements).

TYPE_DOUBLE_V_4 clj

Type: Fixed-size double vector (4 elements).

TYPE_DOUBLE_V_5 clj

Type: Fixed-size double vector (5 elements).

TYPE_DOUBLE_V_6 clj

Type: Fixed-size double vector (6 elements).

TYPE_DOUBLE_V_7 clj

Type: Fixed-size double vector (7 elements).

TYPE_DOUBLE_V_8 clj

Type: Fixed-size double vector (8 elements).

TYPE_DOUBLE_V_9 clj

Type: Fixed-size double vector (9 elements).

TYPE_FLOAT clj

Type: Floating point value.

TYPE_INT clj

Type: Integer value.

TYPE_INT_V_1 clj

Type: Fixed-size integer vector (1 element).

TYPE_INT_V_10 clj

Type: Fixed-size integer vector (10 elements).

TYPE_INT_V_11 clj

Type: Fixed-size integer vector (11 elements).

TYPE_INT_V_12 clj

Type: Fixed-size integer vector (12 elements).

TYPE_INT_V_2 clj

Type: Fixed-size integer vector (2 elements).

TYPE_INT_V_3 clj

Type: Fixed-size integer vector (3 elements).

TYPE_INT_V_4 clj

Type: Fixed-size integer vector (4 elements).

TYPE_INT_V_5 clj

Type: Fixed-size integer vector (5 elements).

TYPE_INT_V_6 clj

Type: Fixed-size integer vector (6 elements).

TYPE_INT_V_7 clj

Type: Fixed-size integer vector (7 elements).

TYPE_INT_V_8 clj

Type: Fixed-size integer vector (8 elements).

TYPE_INT_V_9 clj

Type: Fixed-size integer vector (9 elements).

TYPE_KEYWORD clj

Type: Clojure keyword (rendered as string, keywordized on parse).

TYPE_REF clj

Type: Reference to another spec.

TYPE_STRING clj

Type: String value.

TYPE_STRING_V_1 clj

Type: Fixed-size string vector (1 element).

TYPE_STRING_V_10 clj

Type: Fixed-size string vector (10 elements).

TYPE_STRING_V_11 clj

Type: Fixed-size string vector (11 elements).

TYPE_STRING_V_12 clj

Type: Fixed-size string vector (12 elements).

TYPE_STRING_V_2 clj

Type: Fixed-size string vector (2 elements).

TYPE_STRING_V_3 clj

Type: Fixed-size string vector (3 elements).

TYPE_STRING_V_4 clj

Type: Fixed-size string vector (4 elements).

TYPE_STRING_V_5 clj

Type: Fixed-size string vector (5 elements).

TYPE_STRING_V_6 clj

Type: Fixed-size string vector (6 elements).

TYPE_STRING_V_7 clj

Type: Fixed-size string vector (7 elements).

TYPE_STRING_V_8 clj

Type: Fixed-size string vector (8 elements).

TYPE_STRING_V_9 clj

Type: Fixed-size string vector (9 elements).

UNION clj

Field option: Set of allowed nil types (used internally for optional fields).

user clj

Creates a user message, optionally with images. See internal.llm for details.

validate-data clj

Validates parsed data against a spec. See spec namespace for details.

VALUES clj

Field option: Enum values as map {value description}.