Borrowed EYES for a blind target model.
A session routes to ONE model, and that model may have no :vision capability
(a Copilot tier without vision, glm-5-turbo, deepseek, most coding plans). The
images are still real — the user attached a screenshot, an earlier iteration
plotted a figure — so today they are dropped from the wire and the model is told
to open them with PIL. PIL answers (1920, 1080) RGB and nothing else: pixel
size is not meaning, and the agent burns a tool call to learn nothing.
This namespace closes that gap WITHOUT switching the turn's model: one cheap
side-channel ask! prefers a :vision model from the foreground provider, then
the last provider whose eyes worked, then the cheapest+fastest eyes elsewhere in
the same fleet. Svar owns that ordered provider preference and capability filter.
The pinned model, its thinking chain and its tool continuity are untouched.
Three properties make it affordable rather than wasteful:
A LEAF: svar + attachments + config + runtime-settings + toggles, never back on the loop.
Borrowed EYES for a blind target model. A session routes to ONE model, and that model may have no `:vision` capability (a Copilot tier without vision, glm-5-turbo, deepseek, most coding plans). The images are still real — the user attached a screenshot, an earlier iteration plotted a figure — so today they are dropped from the wire and the model is told to open them with PIL. PIL answers `(1920, 1080) RGB` and nothing else: pixel size is not meaning, and the agent burns a tool call to learn nothing. This namespace closes that gap WITHOUT switching the turn's model: one cheap side-channel `ask!` prefers a `:vision` model from the foreground provider, then the last provider whose eyes worked, then the cheapest+fastest eyes elsewhere in the same fleet. Svar owns that ordered provider preference and capability filter. The pinned model, its thinking chain and its tool continuity are untouched. Three properties make it affordable rather than wasteful: - CONTENT-KEYED CACHE. Attachments replay on every request for the rest of the session, so a per-request description would be re-billed forever. Keyed by the digest of the bytes, an image is described exactly ONCE per process. - OWN FAILURE POLICY. Like titling, the describe call never waits out a 429 and never fails over provider chains: the foreground turn owns the quota. A refusal or a deadline returns nil and the caller keeps today's behaviour. - SECOND-HAND BY CONTRACT. The description is labelled as another model's report in the prompt, never as the agent's own sight, so pixel-exact work still goes through the imaging path. A LEAF: svar + attachments + config + runtime-settings + toggles, never back on the loop.
(available? router)True when the fallback is on AND some configured model can actually see.
True when the fallback is on AND some configured model can actually see.
(blind-model-names)Model NAMES no request may show an image to.
Model NAMES no request may show an image to.
(blind-provider-ids)Provider ids no request may carry an image to.
Provider ids no request may carry an image to.
(clear-cache!)Drop every memoized description. Tests only.
Drop every memoized description. Tests only.
(clear-image-blind!)Forget everything learned about which eyes work, and re-arm the store read. Tests only:
the file itself is left alone, so a test that redirects config/state-path never
touches the machine's real memory.
Forget everything learned about which eyes work, and re-arm the store read. Tests only: the file itself is left alone, so a test that redirects `config/state-path` never touches the machine's real memory.
(describe-attachments router context attachments)(describe-attachments router context attachments preferred-provider)Describe raw image attachments, preferring preferred-provider without pinning to it.
Describe raw image attachments, preferring `preferred-provider` without pinning to it.
(describe-images router context images)(describe-images router context images preferred-provider)Describe wired images, preferring preferred-provider without pinning to it.
Describe wired images, preferring `preferred-provider` without pinning to it.
(descriptions-message described)The plain-TEXT {:role "user"} message that stands in for images a blind target
cannot be shown, or nil when nothing was described.
described is [{:label … :text … :model …}]. The copy is deliberate: the
description is attributed to the model that produced it and marked second-hand,
because an agent that believes it SAW the pixels will happily testify about
details no one described. Pixel-exact work still goes through the bytes.
The plain-TEXT `{:role "user"}` message that stands in for images a blind target
cannot be shown, or nil when nothing was described.
`described` is `[{:label … :text … :model …}]`. The copy is deliberate: the
description is attributed to the model that produced it and marked second-hand,
because an agent that believes it SAW the pixels will happily testify about
details no one described. Pixel-exact work still goes through the bytes.(enabled?)Whether the vision-description fallback may run at all.
Whether the vision-description fallback may run at all.
(forget-working-eye! {:keys [provider-id model]})Drop the remembered eye when it is the one that just failed.
Called for EVERY failure, not only a permanent refusal: an expired credential is no fact about the model, but it does mean this provider is not the one to try FIRST any more. Whoever answers the next image is remembered in its place — and the drop is written through, so a restart cannot resurrect the endpoint that just broke.
Drop the remembered eye when it is the one that just failed. Called for EVERY failure, not only a permanent refusal: an expired credential is no fact about the model, but it does mean this provider is not the one to try FIRST any more. Whoever answers the next image is remembered in its place — and the drop is written through, so a restart cannot resurrect the endpoint that just broke.
(image-blind-model? model)True when model already proved it cannot read pixels, on any provider.
True when `model` already proved it cannot read pixels, on any provider.
(image-blind-provider? provider-id)True when provider-id already proved its wire cannot carry an image.
True when `provider-id` already proved its wire cannot carry an image.
(remember-image-blind! provider-id)Record provider-id as unable to carry an image content part at all. Idempotent, and
written through to the store — the next session starts already knowing.
Record `provider-id` as unable to carry an image content part at all. Idempotent, and written through to the store — the next session starts already knowing.
(remember-image-blind-model! model provider-id)Record model as unable to READ pixels, as learned on provider-id. Idempotent, and
written through to the store.
Deliberately NOT a provider verdict: the same endpoint keeps serving whatever else it has eyes for, and the name stays out everywhere it is offered.
Record `model` as unable to READ pixels, as learned on `provider-id`. Idempotent, and written through to the store. Deliberately NOT a provider verdict: the same endpoint keeps serving whatever else it has eyes for, and the name stays out everywhere it is offered.
(remember-image-refusal! {:keys [provider model scope]})Learn ONE provider-error/image-rejections row at the scope it actually proves:
a wire that has no image variant blinds the PROVIDER, a model that cannot read
pixels blinds only that NAME.
Learn ONE `provider-error/image-rejections` row at the scope it actually proves: a wire that has no image variant blinds the PROVIDER, a model that cannot read pixels blinds only that NAME.
(remember-working-eye! provider-id model)Remember the provider/model pair that just described an image. Idempotent.
The id is kept exactly as svar reported it — provider ids travel as keywords through routing, and a stringified one would silently never match a preference or an exclusion.
Written through to the store, but not once per image: the same pair proving itself
again only reaches disk after MEMORY_REFRESH_MS.
Remember the provider/model pair that just described an image. Idempotent. The id is kept exactly as svar reported it — provider ids travel as keywords through routing, and a stringified one would silently never match a preference or an exclusion. Written through to the store, but not once per image: the same pair proving itself again only reaches disk after `MEMORY_REFRESH_MS`.
(sighted-model router)(sighted-model router preferred-provider)Descriptor of the model this fleet would use to LOOK at an image, or nil when no
configured provider carries :vision — one that already refused pixels, on the
wire or by name, does not count. preferred-provider gets first refusal without
becoming a pin. Cheap: router arithmetic, no I/O.
TOTAL BY CONTRACT. The probe runs INSIDE request assembly and its answer is only
ever an OFFER — nil means the caller keeps today's blind behaviour, so nothing
here is worth failing a turn over. Two ways it can fail on a healthy session:
svar's resolver reads live provider state and throws on provider/model
combinations it rejects, and Vis passes router-SHAPED config maps around (its own
resolve-effective-model is structural for exactly that reason). A probe that
propagated would abort turns that carry no images at all.
Descriptor of the model this fleet would use to LOOK at an image, or nil when no configured provider carries `:vision` — one that already refused pixels, on the wire or by name, does not count. `preferred-provider` gets first refusal without becoming a pin. Cheap: router arithmetic, no I/O. TOTAL BY CONTRACT. The probe runs INSIDE request assembly and its answer is only ever an OFFER — nil means the caller keeps today's blind behaviour, so nothing here is worth failing a turn over. Two ways it can fail on a healthy session: svar's resolver reads live provider state and throws on provider/model combinations it rejects, and Vis passes router-SHAPED config maps around (its own `resolve-effective-model` is structural for exactly that reason). A probe that propagated would abort turns that carry no images at all.
Feature toggle gating the whole side-channel (registered in toggles).
Feature toggle gating the whole side-channel (registered in `toggles`).
(working-eye)The provider/model pair that last described an image, with the stamp of when it proved it — from this process or from the store.
The provider/model pair that last described an image, with the stamp of when it proved it — from this process or from the store.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |