User-message image, video, audio, document and gzip attachments.
Dropping a file onto the terminal pastes its PATH into the input (the terminal's drop behavior — same mechanism pi relies on). At turn start the engine scans the user message for path-shaped tokens that resolve to real image files, reads them, and attaches them to the initial user message as multimodal content blocks. Channel-neutral: every channel gets the same behavior because the scan runs in the engine, not the channel.
What a file IS is sniffed from magic bytes (pi-parity: jpeg / non-animated
png / gif / webp / bmp, plus MP4/QuickTime clips, mp3/m4a/wav/ogg/flac
recordings, PDF/HTML documents and gzip streams) or, for SVG, from the
markup head. Markdown uploads use a filename or MIME hint plus UTF-8 validation;
an extension alone never blesses a payload. A container the model cannot consume
is kept for the HUMAN and NAMED to the model (model-blind-media-type?).
Storing and SENDING are deliberately separate concerns:
max-image-bytes
is skipped with a reason the prompt assembler surfaces.wire-image is the one gate every image crosses on its way to
a provider: it decodes the payload to prove it is pixels, re-containers
what no wire accepts (BMP/SVG -> PNG via image-convert, on
com.blockether/imaging, so it behaves identically in the native image)
and REFUSES what it cannot turn into a picture.The split is what makes a bad attachment survivable. Attachments replay on every later turn, so a row blessed once on the way IN is shipped forever — one corrupt PNG that way is a permanent provider 400. Judged on the way OUT, the same row is simply dropped and the session keeps working.
User-message image, video, audio, document and gzip attachments.
Dropping a file onto the terminal pastes its PATH into the input (the
terminal's drop behavior — same mechanism pi relies on). At turn start
the engine scans the user message for path-shaped tokens that resolve
to real image files, reads them, and attaches them to the initial user
message as multimodal content blocks. Channel-neutral: every channel
gets the same behavior because the scan runs in the engine,
not the channel.
What a file IS is sniffed from magic bytes (pi-parity: jpeg / non-animated
png / gif / webp / bmp, plus MP4/QuickTime clips, mp3/m4a/wav/ogg/flac
recordings, PDF/HTML documents and gzip streams) or, for SVG, from the
markup head. Markdown uploads use a filename or MIME hint plus UTF-8 validation;
an extension alone never blesses a payload. A container the model cannot consume
is kept for the HUMAN and NAMED to the model ([[model-blind-media-type?]]).
Storing and SENDING are deliberately separate concerns:
* STORE — the original bytes under their sniffed container, nothing
converted, nothing downscaled, nothing re-compressed. What the user
supplied is what the session keeps, and a file over `max-image-bytes`
is skipped with a reason the prompt assembler surfaces.
* SEND — [[wire-image]] is the one gate every image crosses on its way to
a provider: it decodes the payload to prove it is pixels, re-containers
what no wire accepts (BMP/SVG -> PNG via `image-convert`, on
`com.blockether/imaging`, so it behaves identically in the native image)
and REFUSES what it cannot turn into a picture.
The split is what makes a bad attachment survivable. Attachments replay on
every later turn, so a row blessed once on the way IN is shipped forever —
one corrupt PNG that way is a permanent provider 400. Judged on the way OUT,
the same row is simply dropped and the session keeps working.(attachment-audience attachment)This attachment's audience word, normalized — "both" when unstamped.
A model-blind-media-type? artifact is CLAMPED to "user" here rather
than at each call site: documents, gzip files and recordings are never image
blocks, and every gate that asks this question inherits the refusal here.
This attachment's audience word, normalized — `"both"` when unstamped. A [[model-blind-media-type?]] artifact is CLAMPED to `"user"` here rather than at each call site: documents, gzip files and recordings are never image blocks, and every gate that asks this question inherits the refusal here.
The CLOSED vocabulary of an attachment's AUDIENCE — who the artifact is for:
"both" (the default) paints it for the human AND sends the image to
the model."user" paints it for the human and keeps the bytes off the wire."model" sends it to the model and paints nothing for the human.One word, three routes; there is no fourth and no second copy of this table.
The CLOSED vocabulary of an attachment's AUDIENCE — who the artifact is for:
* `"both"` (the default) paints it for the human AND sends the image to
the model.
* `"user"` paints it for the human and keeps the bytes off the wire.
* `"model"` sends it to the model and paints nothing for the human.
One word, three routes; there is no fourth and no second copy of this table.ISO-BMFF major brands that are a RECORDING, not a clip. A voice memo is an
.m4a -- the very container an MP4 uses -- so a naive ftyp check files it
as a video, and the send-time gate then tries to sample frames out of a file
that has none.
ISO-BMFF major brands that are a RECORDING, not a clip. A voice memo is an `.m4a` -- the very container an MP4 uses -- so a naive `ftyp` check files it as a video, and the send-time gate then tries to sample frames out of a file that has none.
(audio-media-type? media-type)True when media-type is one of audio-media-types.
True when `media-type` is one of [[audio-media-types]].
Audio containers vis stores. No wire carries any of them: a recording is not
pixels, so it is kept for the HUMAN and the model is TOLD the file is there
(model-blind-media-type?) instead of being handed bytes it cannot hear.
Audio containers vis stores. No wire carries any of them: a recording is not pixels, so it is kept for the HUMAN and the model is TOLD the file is there ([[model-blind-media-type?]]) instead of being handed bytes it cannot hear.
(collect-user-images text)(collect-user-images text
{:keys [workspace-root max-bytes max-images]
:or {max-bytes max-image-bytes
max-images max-image-count}})Scan text (one user message) for paths of readable image files and
load them as attachments.
Options:
:workspace-root - base for relative candidates (default: cwd).
:max-bytes - per-image cap (default max-image-bytes).
:max-images - attachment count cap (default max-image-count).
Returns {:attached [{:path :media-type :base64 :size :size-label}] :skipped [{:path :reason}]} — :skipped names sniffed
image files that were rejected (size cap / count cap) so the prompt
assembler can tell the model WHY an image it can see referenced is
absent. Non-image or non-existent candidates are silently ignored.
Never throws — a failure to read one file skips that file.
Scan `text` (one user message) for paths of readable image files and
load them as attachments.
Options:
:workspace-root - base for relative candidates (default: cwd).
:max-bytes - per-image cap (default [[max-image-bytes]]).
:max-images - attachment count cap (default [[max-image-count]]).
Returns `{:attached [{:path :media-type :base64 :size :size-label}]
:skipped [{:path :reason}]}` — `:skipped` names sniffed
image files that were rejected (size cap / count cap) so the prompt
assembler can tell the model WHY an image it can see referenced is
absent. Non-image or non-existent candidates are silently ignored.
Never throws — a failure to read one file skips that file.(detect-audio-mime b)Sniff a supported audio MIME type from the leading bytes of a file. Returns "audio/mpeg" | "audio/aac" | "audio/mp4" | "audio/wav" | "audio/aiff" | "audio/x-caf" | "audio/amr" | "audio/ogg" | "audio/flac", or nil.
Magic bytes only, like every other sniff here: an extension is a CLAIM, and
a phone that hands the webview a .m4a with no MIME type at all is the
normal case rather than the odd one. The four beyond the obvious ones are all
RECORDERS rather than music: caf and aiff are what an iPhone writes,
amr and bare aac what an Android one does.
Sniff a supported audio MIME type from the leading bytes of a file. Returns "audio/mpeg" | "audio/aac" | "audio/mp4" | "audio/wav" | "audio/aiff" | "audio/x-caf" | "audio/amr" | "audio/ogg" | "audio/flac", or nil. Magic bytes only, like every other sniff here: an extension is a CLAIM, and a phone that hands the webview a `.m4a` with no MIME type at all is the normal case rather than the odd one. The four beyond the obvious ones are all RECORDERS rather than music: `caf` and `aiff` are what an iPhone writes, `amr` and bare `aac` what an Android one does.
(detect-document-mime b)Sniff a PDF, HTML or XHTML document from its bytes, never its extension.
Sniff a PDF, HTML or XHTML document from its bytes, never its extension.
(detect-gzip-mime b)Sniff the RFC 1952 magic bytes and return the canonical gzip media type.
Sniff the RFC 1952 magic bytes and return the canonical gzip media type.
(detect-image-mime b)Sniff a supported image MIME type from the leading bytes of a file.
Returns "image/png" | "image/jpeg" | "image/gif" | "image/webp" |
"image/bmp" | "image/svg+xml", or nil when the bytes are not a supported
still image. BMP and SVG are not wire-legal themselves — they are readable,
and image-convert turns them into PNG before they are sent.
Animated PNGs and JPEG-LS return nil (provider-rejected shapes).
Sniff a supported image MIME type from the leading bytes of a file. Returns "image/png" | "image/jpeg" | "image/gif" | "image/webp" | "image/bmp" | "image/svg+xml", or nil when the bytes are not a supported still image. BMP and SVG are not wire-legal themselves — they are readable, and `image-convert` turns them into PNG before they are sent. Animated PNGs and JPEG-LS return nil (provider-rejected shapes).
(detect-jsonl-mime b)Recognize a complete leading JSON object in a bounded file head. The remainder may be truncated by sniffing; original log bytes are stored without parsing.
Recognize a complete leading JSON object in a bounded file head. The remainder may be truncated by sniffing; original log bytes are stored without parsing.
(detect-media-mime b)The sniffed type of anything vis can attach: detect-image-mime first,
then audio, video, document and gzip detection. Stills win the ISO-BMFF tie
deliberately; recordings are asked about before clips for the same reason.
The sniffed type of anything vis can attach: [[detect-image-mime]] first, then audio, video, document and gzip detection. Stills win the ISO-BMFF tie deliberately; recordings are asked about before clips for the same reason.
(detect-video-mime b)Sniff a supported video MIME type from the leading bytes of a file. Returns "video/mp4" | "video/quicktime", or nil.
ISO-BMFF only (ftyp at offset 4); the major brand at offset 8 decides.
qt is QuickTime, a [[still-image-brands]] brand is a PHOTO and an
audio-brands brand is a RECORDING (both nil, never a clip), anything
else is MP4.
Sniff a supported video MIME type from the leading bytes of a file. Returns "video/mp4" | "video/quicktime", or nil. ISO-BMFF only (`ftyp` at offset 4); the major brand at offset 8 decides. `qt ` is QuickTime, a [[still-image-brands]] brand is a PHOTO and an [[audio-brands]] brand is a RECORDING (both nil, never a clip), anything else is MP4.
The one vocabulary for documents stored for the human and named to the model.
The one vocabulary for documents stored for the human and named to the model.
(gzip-media-type? media-type)True when media-type names a gzip stream from gzip-media-types.
True when `media-type` names a gzip stream from [[gzip-media-types]].
MIME spellings file providers use for a gzip stream. Intake sniffs and stores
the canonical application/gzip; the legacy spelling remains an accepted claim
because Android document providers still return it.
MIME spellings file providers use for a gzip stream. Intake sniffs and stores the canonical `application/gzip`; the legacy spelling remains an accepted claim because Android document providers still return it.
(hidden-from-model? attachment)True when this attachment is for the HUMAN only (audience="user"): shown in
the UI and stored in the session DB, but deliberately never rendered as an
image block.
The escape hatch for the one thing multimodal replay cannot undo — an image
replays IN FULL on every later request, so a screenshot the model does not
actually need is re-billed forever. attach(..., audience='user') stamps
:audience "user", and the send-time gate routes the row to :skipped +
:readable-blind? instead: the model is TOLD the file exists and can open the
bytes on demand (read_attachment) or ask for it back with
show_attachment.
True when this attachment is for the HUMAN only (`audience="user"`): shown in the UI and stored in the session DB, but deliberately never rendered as an image block. The escape hatch for the one thing multimodal replay cannot undo — an image replays IN FULL on every later request, so a screenshot the model does not actually need is re-billed forever. `attach(..., audience='user')` stamps `:audience "user"`, and the send-time gate routes the row to `:skipped` + `:readable-blind?` instead: the model is TOLD the file exists and can open the bytes on demand (`read_attachment`) or ask for it back with `show_attachment`.
(hidden-from-user? attachment)True when this attachment is for the MODEL only (audience="model"): the
bytes ride the request and the human's transcript stays clean. The twin of
hidden-from-model?, and the reason a run can gather evidence for itself
without burying the human in thumbnails it never asked to review.
True when this attachment is for the MODEL only (`audience="model"`): the bytes ride the request and the human's transcript stays clean. The twin of [[hidden-from-model?]], and the reason a run can gather evidence for itself without burying the human in thumbnails it never asked to review.
(human-only-media-type? media-type)True when media-type names a file from human-only-media-types. Media-type
parameters (text/html; charset=utf-8) are dropped before the lookup.
True when `media-type` names a file from [[human-only-media-types]]. Media-type parameters (`text/html; charset=utf-8`) are dropped before the lookup.
Media types a model must NEVER be handed as an image block, no matter what
audience the caller asked for. Documents and gzip streams are not pixels; the
honest route is to keep their bytes in the session, name the file to the model,
and let it inspect those bytes on demand with read_attachment.
Media types a model must NEVER be handed as an image block, no matter what audience the caller asked for. Documents and gzip streams are not pixels; the honest route is to keep their bytes in the session, name the file to the model, and let it inspect those bytes on demand with `read_attachment`.
(image-label {:keys [path filename]})How ONE image attachment is NAMED to the model: its path when it has one, else the filename it was given. The manifest, the skip reason and the vision-fallback description all key off this, so a description and the row it describes cannot drift apart.
How ONE image attachment is NAMED to the model: its path when it has one, else the filename it was given. The manifest, the skip reason and the vision-fallback description all key off this, so a description and the row it describes cannot drift apart.
MIME spellings used by Markdown file providers. Intake stores text/markdown.
MIME spellings used by Markdown file providers. Intake stores `text/markdown`.
Per-recording byte cap. A recording never reaches a provider at all --
it is stored, played back by the human, and NAMED to the model
(model-blind-media-type?) -- so this bounds the original the session
keeps and the memory one drop can cost, nothing on any wire. The same 32MB
ceiling a clip answers to covers a long voice memo or an interview take.
Per-recording byte cap. A recording never reaches a provider at all -- it is stored, played back by the human, and NAMED to the model ([[model-blind-media-type?]]) -- so this bounds the original the session keeps and the memory one drop can cost, nothing on any wire. The same 32MB ceiling a clip answers to covers a long voice memo or an interview take.
Per-image cap on the WIRE PAYLOAD, in bytes of BASE64 -- which is what a
provider measures, not the picture that decodes out of it (see
wire-byte-budget, so the picture itself gets 3/4 of this).
Anthropic's API limit is 5MB/image and it is enforced on the base64 string
(image exceeds 5 MB maximum: 5994492 bytes > 5242880 bytes); OpenAI and
Gemini bound the whole request far more generously. The smallest common
bound keeps one attachment valid on every wire.
Per-image cap on the WIRE PAYLOAD, in bytes of BASE64 -- which is what a provider measures, not the picture that decodes out of it (see `wire-byte-budget`, so the picture itself gets 3/4 of this). Anthropic's API limit is 5MB/image and it is enforced on the base64 string (`image exceeds 5 MB maximum: 5994492 bytes > 5242880 bytes`); OpenAI and Gemini bound the whole request far more generously. The smallest common bound keeps one attachment valid on every wire.
Attachment count cap per user message. Guards against a pathological message (e.g. a pasted directory listing) ballooning the request.
Attachment count cap per user message. Guards against a pathological message (e.g. a pasted directory listing) ballooning the request.
Last-resort cap on ONE STORED artifact, in DECODED bytes (64MB).
Distinct from max-image-bytes (what a PROVIDER accepts on the wire) and
max-upload-image-bytes (what the gateway accepts as a still): this bounds
what ANY writer -- a tool's attach(), a companion revision, an extension --
can put in a single session row, whatever the media type. SQLite refuses a
bound value over SQLITE_MAX_LENGTH (1e9 bytes) with [SQLITE_TOOBIG] and
takes the whole enclosing write down with it, so an unbounded payload is a
way to LOSE an iteration, not merely a way to grow the store.
Last-resort cap on ONE STORED artifact, in DECODED bytes (64MB). Distinct from [[max-image-bytes]] (what a PROVIDER accepts on the wire) and [[max-upload-image-bytes]] (what the gateway accepts as a still): this bounds what ANY writer -- a tool's `attach()`, a companion revision, an extension -- can put in a single session row, whatever the media type. SQLite refuses a bound value over `SQLITE_MAX_LENGTH` (1e9 bytes) with `[SQLITE_TOOBIG]` and takes the whole enclosing write down with it, so an unbounded payload is a way to LOSE an iteration, not merely a way to grow the store.
Per-still INTAKE cap (25MB): the largest image the gateway accepts and the
session stores. Distinct from max-image-bytes, which is the PROVIDER cap
applied to the bytes that actually go on the wire.
Per-still INTAKE cap (25MB): the largest image the gateway accepts and the session stores. Distinct from [[max-image-bytes]], which is the PROVIDER cap applied to the bytes that actually go on the wire.
Per-clip byte cap. A clip is never sent in its own container -- it leaves as a small animated GIF ([[wire-verdict]]) -- so this bounds the ORIGINAL the session keeps and the memory one drop can cost, not any provider limit. A 32MB ceiling covers a normal screen recording or a phone clip.
Per-clip byte cap. A clip is never sent in its own container -- it leaves as a small animated GIF ([[wire-verdict]]) -- so this bounds the ORIGINAL the session keeps and the memory one drop can cost, not any provider limit. A 32MB ceiling covers a normal screen recording or a phone clip.
(model-blind-media-type? media-type)True when these bytes must never ride a request as an image block, whatever
audience the caller asked for: a human-only-media-type? file, or an
audio-media-type? recording. None is pixels, so the honest route is to keep
the bytes for the human and TELL the model the file is there.
True when these bytes must never ride a request as an image block, whatever audience the caller asked for: a [[human-only-media-type?]] file, or an [[audio-media-type?]] recording. None is pixels, so the honest route is to keep the bytes for the human and TELL the model the file is there.
(normalize-audience v)Coerce whatever the shim bridge, a wire payload or a DB row carries into a
member of audiences, defaulting to "both". An unknown word falls back
to the PERMISSIVE default: a typo must not silently hide someone's figure.
Coerce whatever the shim bridge, a wire payload or a DB row carries into a member of [[audiences]], defaulting to `"both"`. An unknown word falls back to the PERMISSIVE default: a typo must not silently hide someone's figure.
How far past max-image-bytes a file may sit and still be worth reading for
a shrink attempt. Bounds the memory one pathological drop can cost.
How far past `max-image-bytes` a file may sit and still be worth reading for a shrink attempt. Bounds the memory one pathological drop can cost.
(prepare-inline-attachments attachments)(prepare-inline-attachments attachments
{:keys [max-bytes max-images]
:or {max-bytes max-image-bytes
max-images max-image-count}})Validate already-encoded attachments delivered INLINE (web/API upload) rather
than as filesystem paths. Each entry is {:base64 :filename :media-type?}; the
base64 may be a bare payload or a data:...;base64, URL. Decodes each, sniffs
the MIME from magic bytes, then checks Markdown hints against valid UTF-8 text.
A declared :media-type alone never accepts binary bytes. Enforces the same caps
as collect-user-images, returning its {:attached [...] :skipped [...]} shape.
Never throws.
The ORIGINAL payload is stored under its validated type and never converted.
Image adaptation remains a SEND-time question (see wire-image); human-only
files stay available through read_attachment.
Validate already-encoded attachments delivered INLINE (web/API upload) rather
than as filesystem paths. Each entry is `{:base64 :filename :media-type?}`; the
base64 may be a bare payload or a `data:...;base64,` URL. Decodes each, sniffs
the MIME from magic bytes, then checks Markdown hints against valid UTF-8 text.
A declared `:media-type` alone never accepts binary bytes. Enforces the same caps
as [[collect-user-images]], returning its `{:attached [...] :skipped [...]}` shape.
Never throws.
The ORIGINAL payload is stored under its validated type and never converted.
Image adaptation remains a SEND-time question (see [[wire-image]]); human-only
files stay available through `read_attachment`.(provider-image-media-type? media-type)True when media-type is one of provider-image-media-types.
True when `media-type` is one of [[provider-image-media-types]].
The ONLY image media types a vision wire accepts VERBATIM. Anthropic names
exactly these four in its rejection (the image data you provided does not represent a valid image … supported image formats: ['image/jpeg', 'image/png', 'image/gif', 'image/webp']); OpenAI and Gemini are supersets.
Anything else — an image/svg+xml figure from attach/matplotlib, a
BMP screenshot — is a hard 400, and since attachments
REPLAY on every later turn ONE such row kills the whole session.
The ONLY image media types a vision wire accepts VERBATIM. Anthropic names exactly these four in its rejection (`the image data you provided does not represent a valid image … supported image formats: ['image/jpeg', 'image/png', 'image/gif', 'image/webp']`); OpenAI and Gemini are supersets. Anything else — an `image/svg+xml` figure from `attach`/matplotlib, a BMP screenshot — is a hard 400, and since attachments REPLAY on every later turn ONE such row kills the whole session.
(scan-image-descriptors text)(scan-image-descriptors text {:keys [workspace-root]})Resolve every image the user text points at, WITHOUT loading pixel bytes.
Returns [{:path :media-type :size :size-label :filename}] for files whose
magic bytes sniff to a supported still image — ordered, de-duped. Cheap
enough to run on every paste (only a small file-head read per candidate).
Never throws.
Resolve every image the user text points at, WITHOUT loading pixel bytes.
Returns `[{:path :media-type :size :size-label :filename}]` for files whose
magic bytes sniff to a supported still image — ordered, de-duped. Cheap
enough to run on every paste (only a small file-head read per candidate).
Never throws.(sniff-file-mime f)Read a file head and return its sniffed attachment media type. nil on any read failure or unsupported bytes; a caller never needs to trust an extension.
Read a file head and return its sniffed attachment media type. nil on any read failure or unsupported bytes; a caller never needs to trust an extension.
(text->chip-preview text)One-line, path-free rendering of ONE user message for compact previews (queue rows, tab titles, notifications).
Every image-shaped path token collapses to a short name.png chip (see
text->inline-chips), the prose around it survives, and whitespace collapses
to single spaces. Pure string work — no filesystem access, no pixel bytes — so
it is safe on a paint path and on text whose files have since been deleted.
That is the whole point: a queued message authored by dropping a screenshot
used to render as a raw /var/folders/…/clipboard-….png, which tells the user
nothing about what they queued.
Returns nil when the message is nothing but images — the caller should paint attachment chips alone rather than an empty row. The ORIGINAL text is never mutated; callers keep it for re-send/edit so the paths still re-attach.
One-line, path-free rendering of ONE user message for compact previews (queue rows, tab titles, notifications). Every image-shaped path token collapses to a short `name.png` chip (see `text->inline-chips`), the prose around it survives, and whitespace collapses to single spaces. Pure string work — no filesystem access, no pixel bytes — so it is safe on a paint path and on text whose files have since been deleted. That is the whole point: a queued message authored by dropping a screenshot used to render as a raw `/var/folders/…/clipboard-….png`, which tells the user nothing about what they queued. Returns nil when the message is nothing but images — the caller should paint attachment chips alone rather than an empty row. The ORIGINAL text is never mutated; callers keep it for re-send/edit so the paths still re-attach.
(text->inline-chips text)text with every image-shaped PATH token replaced by a short name.png
chip, and NOTHING else touched — line breaks, indentation and the prose around
each path all survive.
Pure string work: no filesystem access, no pixel bytes, so it is safe on a paint path and on text whose files have since been deleted. Use it wherever a user message is DISPLAYED (transcript bubble, notification body); the original text is never mutated, so re-send/edit still ships the path that re-attaches the picture.
`text` with every image-shaped PATH token replaced by a short `name.png` chip, and NOTHING else touched — line breaks, indentation and the prose around each path all survive. Pure string work: no filesystem access, no pixel bytes, so it is safe on a paint path and on text whose files have since been deleted. Use it wherever a user message is DISPLAYED (transcript bubble, notification body); the original text is never mutated, so re-send/edit still ships the path that re-attaches the picture.
(unsupported-media-reason media-type)Why an otherwise readable image was not attached, in the user's words.
Why an otherwise readable image was not attached, in the user's words.
How far past max-image-bytes a STILL may sit and still be accepted for
storage. A phone photo or a retina screenshot is routinely tens of megabytes
while the provider ceiling is 5MB, and wire-image shrinks whatever it is
given on the way OUT — so refusing the upload only loses the picture.
How far past `max-image-bytes` a STILL may sit and still be accepted for storage. A phone photo or a retina screenshot is routinely tens of megabytes while the provider ceiling is 5MB, and [[wire-image]] shrinks whatever it is given on the way OUT — so refusing the upload only loses the picture.
(video-media-type? media-type)True when media-type is one of video-media-types.
True when `media-type` is one of [[video-media-types]].
Video containers vis stores. No wire takes any of them verbatim: a clip is transcoded to an animated GIF at SEND time (see [[wire-verdict]]), which is why storing one is safe even though sending the container never is.
Video containers vis stores. No wire takes any of them verbatim: a clip is transcoded to an animated GIF at SEND time (see [[wire-verdict]]), which is why storing one is safe even though sending the container never is.
(wire-image attachment)(wire-image {:keys [media-type base64] :as attachment}
{:keys [max-bytes max-dimension]
:or {max-bytes max-image-bytes
max-dimension image-convert/max-wire-dimension}})ONE stored attachment as the wire will actually carry it — the single gate every image crosses on its way to a provider, applied at SEND time.
Send time is the only correct layer. Storage is permanent and providers are
not: the model that will read a row is unknown when the row is written, an
attachment REPLAYS on every later turn, and a payload converted (or blessed)
on the way IN can never be reconsidered — which is exactly how one corrupt
83-byte PNG, wire-legal by media type and garbage in its IDAT, bricked a
whole session with a permanent Could not process image 400. Here the same
row is re-judged every turn, so a bad one is DROPPED instead of fatal and the
session heals itself.
Every container is handled in one place:
max-dimension — DOWNSCALED to fit, because a provider
refuses an oversized picture outright once a request carries many images
(image-convert/max-wire-dimension), whatever it weighsmax-bytes — OPTIMIZED first (the
imaging library's oxipng / jpegtran / gifsicle pass, then its bounded
ladder) and refused only when even that does not fit. The cap is on the
encoded string every wire actually weighs, so the picture is held to 3/4
of itReturns the attachment with :media-type/:base64/:size/:size-label
replaced by the WIRE payload, {:path :filename :reason} when it cannot be
sent, or nil when it is not an image at all. Never throws.
ONE stored attachment as the wire will actually carry it — the single gate
every image crosses on its way to a provider, applied at SEND time.
Send time is the only correct layer. Storage is permanent and providers are
not: the model that will read a row is unknown when the row is written, an
attachment REPLAYS on every later turn, and a payload converted (or blessed)
on the way IN can never be reconsidered — which is exactly how one corrupt
83-byte PNG, wire-legal by media type and garbage in its `IDAT`, bricked a
whole session with a permanent `Could not process image` 400. Here the same
row is re-judged every turn, so a bad one is DROPPED instead of fatal and the
session heals itself.
Every container is handled in one place:
* JPEG / PNG / GIF / WebP — decoded to prove they are pixels, then sent
BYTE-IDENTICAL (a header sniff is not a picture)
* SVG / SVGZ — rasterized to PNG (no wire reads markup)
* MP4 / QuickTime — sampled into an animated GIF (no wire takes a clip)
* BMP, TIFF, anything else the decoder reads — re-containered to PNG
* HEIC/AVIF, a corrupt raster, an image past the decoder's limits —
refused, with the decoder's own words
* any side over `max-dimension` — DOWNSCALED to fit, because a provider
refuses an oversized picture outright once a request carries many images
(`image-convert/max-wire-dimension`), whatever it weighs
* anything whose BASE64 payload passes `max-bytes` — OPTIMIZED first (the
imaging library's oxipng / jpegtran / gifsicle pass, then its bounded
ladder) and refused only when even that does not fit. The cap is on the
encoded string every wire actually weighs, so the picture is held to 3/4
of it
* a non-image artifact (csv/pdf/wav/…) — nil: not an image, not a failure
Returns the attachment with `:media-type`/`:base64`/`:size`/`:size-label`
replaced by the WIRE payload, `{:path :filename :reason}` when it cannot be
sent, or nil when it is not an image at all. Never throws.(wire-images images)(wire-images images {:keys [vision?] :or {vision? true} :as opts})wire-image over a whole user message's attachments, keeping the
{:attached [...] :skipped [{:path :reason}]} shape the prompt manifest
speaks — so an image that could not be sent is NAMED to the model instead of
vanishing.
:vision? false (a text-only target: Copilot without vision, glm-5-turbo,
deepseek …) attaches nothing and marks every image :readable-blind?: the
files are real and on disk, so the manifest can tell the model to open them
with an imaging library instead of hunting for blocks that are not there.
A HUMAN-ONLY row (hidden-from-model?) takes that same blind path even on a
vision model, and is checked FIRST: the caller asked for the bytes to stay
off the wire, and that beats every capability question.
[[wire-image]] over a whole user message's attachments, keeping the
`{:attached [...] :skipped [{:path :reason}]}` shape the prompt manifest
speaks — so an image that could not be sent is NAMED to the model instead of
vanishing.
`:vision?` false (a text-only target: Copilot without vision, glm-5-turbo,
deepseek …) attaches nothing and marks every image `:readable-blind?`: the
files are real and on disk, so the manifest can tell the model to open them
with an imaging library instead of hunting for blocks that are not there.
A HUMAN-ONLY row ([[hidden-from-model?]]) takes that same blind path even on a
vision model, and is checked FIRST: the caller asked for the bytes to stay
off the wire, and that beats every capability question.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |