toolnexus.classifier — net.clojars.muthuishere/toolnexus 0.18.0

canonical-request clj/s

(canonical-request model questions)

The bytes the byte-identity claim covers: model + questions, keys sorted recursively in ASCII (code-point) order, arrays NEVER reordered, compact separators, and < > & ' " plus non-ASCII transmitted RAW.

koine.json/write-str already is that encoder — it owns key order, escaping and number formatting precisely so the two hosts cannot drift — so there is no second canonicaliser here to keep in agreement with it.

state is DELIBERATELY NOT HERE. It is transmitted verbatim as the host supplied it and is outside the claim, because numbers do not canonicalise across languages (-0.0 renders four ways across our own seven runtimes). Do not re-widen this: a caller who needs their state pinned canonicalises it themselves before handing it over.

choice clj/s

(choice decision key)

The choice answer at key, or an error.

choice-over clj/s

(choice-over instructions items)

A choice over any (name, description) pairs — a Tool, a skill, an agent, an A2A card skill. The description must say what picking that option would MEAN; see the encoding obligation on choice-question.

Keys may be strings, keywords or symbols; each reaches the wire as its plain name, with no : sigil (§8B).

choice-question clj/s

(choice-question instructions criteria)

One option from a named set, 1..255 options. criteria maps an option id to WHAT PICKING IT WOULD MEAN.

THE ENCODING OBLIGATION IS THE CALLER'S (§8B, docs/adr/0021 D1). criteria[id] is the ONLY thing that differentiates one option from another to the model: the instructions describe the question and the state describes the situation, and neither says what picking left rather than right would mean. Passing the id itself, an empty string, or one value repeated is schema-valid, passes validation, returns HTTP 200 and a well-formed distribution — and ranks at chance (measured: 17 apples described by consequence, 0/1/0 described by id, against a shuffle control's 1/0/1).

create-classifier clj/s

(create-classifier opts)

Build a classifier. Options mirror §8 create-client field-for-field wherever a field makes sense, so a host that has configured one has configured the other:

:style "systemone" | "llm" | "custom" | "static" (default "systemone") :base-url default "https://api.typesafe.ai/v1" :model default "jev-latest" :api-key-env the NAME of an env var, never a value — read at call time and never logged. Default "TYPESAFE_API_KEY". §8's :api-key takes a value; this option deliberately does not. :headers extra request headers; values expand ${ENV_VAR} from the environment AT CALL TIME and are NEVER logged, identically to remote-MCP headers (§2) :timeout-ms per REQUEST, not per run — default 10000 :http-client the §8 injectable transport, (fn [url headers body]); scope is the classifier path only :retry-base-ms base of the retry backoff in ms (default 500); the delay is base * 2^attempt with no jitter, and a Retry-After header still wins. :retries transient-failure budget (default 2); retries on 408/429/500/502/503/504/529 + network. Widen the status set with :retryable-statuses. :retryable-statuses extra HTTP statuses to treat as retryable, ADDED to the default set (429/500/502/503/504/529, plus 408 here). It can only widen: a host cannot remove 429 and lose Retry-After handling with it. This sets the DEFAULT classification; :on-error still runs per attempt and has the final say, so :on-error returning :fail overrides a status listed here. Example: a Cloudflare-fronted origin that answers 520–527. :on-error the §8 ErrorInfo -> :retry | :fail classifier, REUSED verbatim from toolnexus.client along with the Retry-After delay-seconds rule. There is no second retry policy here and no :suspend tier. :request-params extra top-level body keys, shallow-merged AFTER the classifier builds its own; a param WINS on collision :body-transform runs LAST on the assembled body; its return value is sent :on-metric the SAME §8 sink — "classifier.evaluate" events and the degenerate-criteria "classifier.warning" :client :style "llm" only — the §8 client to emulate over :evaluate :style "custom" only — (fn [state questions] decision); every wire option is ignored :decisions :style "static" only — a vector of {:state … :questions … :response <parsed body>}

A style whose required option is missing, and an unknown style, are rejected HERE rather than at the first call.

default-api-key-env clj/s

The NAME of the environment variable holding the credential, never a value.

default-base-url clj/s

The System One endpoint base. OpenRouter (https://openrouter.ai/api/v1) serves this wire today; self-hosted and open-weights implementations speak it too.

default-model clj/s

The floating alias. PIN IT (e.g. "jev-1.13.0") once thresholds are tuned — :model on the decision echoes what actually answered, which may be more specific.

default-timeout-ms clj/s

Bounds ONE request. A classifier has no loop to bound.

degenerate-criteria clj/s

(degenerate-criteria criteria)

The §8B predicate. Returns a human reason string when a choice's criteria are degenerate, nil otherwise. Degenerate <=> ANY of:

1. every value is empty (empty string or absent), or
2. every value equals its own key, or
3. every value is identical to every other value (n >= 2).

A single-option choice (n = 1) is NEVER reported — there is nothing to differentiate. Note the ordering: with n = 1 rules 1 and 2 can still hold and rule 3 is vacuous, so the n < 2 gate comes first for all three.

evaluate clj/s

(evaluate c state questions)

A state plus typed questions in, a decision out.

state is a string, a map, or a vector — whatever the host already has. questions maps CALLER-CHOSEN keys to question maps. The keys are ADDRESSING, NOT CONTENT: they are never transmitted to the model, so a key MAY be a tool, skill or agent name verbatim, and two evaluations differing only in their keys send identical content.

Questions are INDEPENDENT — one answer is never context for another. A backend that cannot guarantee that reports :calibrated false, which carries both caveats.

levels clj/s

(levels answer)

A score answer's legend in LEVEL order, which the map itself loses. Sorted numerically-by-length-then-lexicographically, because "2" < "10" as levels and "10" < "2" as strings.

max-choice-options clj/s

The client-side cap on a choice's named options (§8B).

max-score-levels clj/s

metric-evaluate clj/s

The :event value of the per-evaluation metric. It goes into the SAME §8 :on-metric sink as the client's "llm" event and is NOT folded into any registry, so rendered metrics text is unchanged by this file's existence.

metric-warning clj/s

The :event value of the degenerate-criteria warning. Carries :question.

min-score-levels clj/s

near-uniform-tolerance clj/s

The ABSOLUTE tolerance on max|p - 1/n|, compared INCLUSIVELY (§8B). Pinned across every port; examples/judge/near-uniform.json pins both sides at 0.0499 / 0.0501, so no port needs an epsilon.

near-uniform? clj/s

(near-uniform? probabilities)

Whether a choice answer's probability map is indistinguishable from flat:

nearUniform  <=>  max over i of |p_i - 1/n|  <=  0.05

n is the number of ENTRIES IN THE MAP and the values are taken AS RETURNED — not renormalised, not sorted, not rounded; an offered option absent from the map counts as 0 by not being an entry. The tolerance is ABSOLUTE (a relative band collapses below the wire's two-decimal rounding on a 255-option roster) and the comparison is INCLUSIVE. n = 1 is trivially uniform; an EMPTY map has no distribution at all and is false.

DERIVED ON DECODE, never read from the wire: no wire change, no request change, no fixture change.

ADVISORY, NOT A CORRECTNESS SIGNAL. It detects an encoding that gave the model nothing to rank on — the one encoding health check available with no ground truth. It cannot tell a good encoding from a subtly wrong one, because a wrong-but-answerable question still reads as answerable. :calibrated carries the same caveat.

noul clj/s

(noul decision key)

The noul answer at key, or an error.

noul-question clj/s

(noul-question instructions)

(noul-question instructions criteria)

The probability that a statement holds, one number in 0..1. It reports NO confidence: the number IS the answer.

criteria, when given, is {"true" <desc> "false" <desc>}. Omit the arity rather than passing nil or {} — the three are different values and the wire carries the difference.

parse-decision clj/s

(parse-decision body)

A recorded or live response body (already parsed JSON with STRING keys) as a Decision:

{:model      what actually answered
 :answers    {caller-key -> answer}
 :usage      {:input-tokens :output-tokens :cost}
 :calibrated true|false}

calibrated ABSENT means true: the systemone wire reports calibration by being itself, and a backend that is not calibrated says so explicitly. A THRESHOLD TUNED AGAINST ONE BACKEND DOES NOT TRANSFER TO ANOTHER.

score clj/s

(score decision key)

The score answer at key, or an error.

score-question clj/s

(score-question instructions criteria)

A rating against an ORDERED rubric of 2..10 levels. criteria is a VECTOR and its order IS the level numbering, so it is never sorted — a "sort everything" canonicaliser silently renumbers the rubric.

styles clj/s

The four backends (§8B). :systemone is the default; :static is what CI runs — no network, no credential.