Container conversion for attachment payloads -- NOT optimization.
vis stores and replays the attachment PICTURE verbatim: whatever the user
dropped is what is stored and what the provider sees, pixel for pixel (the
stored ENCODING may be shrunk losslessly -- compact). The one thing that
cannot be passed through is a container no vision wire accepts (attachments/ provider-image-media-types): a BMP is a perfectly good image and an SVG is
a perfectly good figure, and both are a hard 400 on every provider. Since an
attachment REPLAYS on every later turn, one such row kills the whole session.
So this namespace does exactly one thing: turn bytes a provider REFUSES into the same picture in a container it ACCEPTS.
.svg / gzipped .svgz) -> rendered PNG.mp4 / .mov) -> animated GIFEverything a renderer can answer is ASKED, not re-implemented.
com.blockether/imaging (Rust image + resvg over FFM) decodes, sniffs,
gunzips .svgz, resolves width="100%"/cm/viewBox letterboxing, and
rasterizes. It also owns the two BROWSER repairs resvg alone refuses, behind
imaging/svg-canvas:
viewBox.Both used to be hand-written Clojure HERE. They are Rust in the library now --
one implementation for every caller, and vis only asks for the answer.
renderer-delegation-test still pins that answer document by document, so a
change in resvg or in the repair shows up as a vis test failure.
Conversion is 1:1 and never re-compresses what the wire already takes: no
quality knob, no size heuristics on the way through. There are exactly two
exceptions, both narrow and both named: fit-within, which the send gate
calls ONLY for a payload over the wire's byte cap that would otherwise be
DROPPED, and compact, which the storage rail calls on the way into the
session_attachment BLOB and which is LOSSLESS -- same picture, fewer bytes,
verified by a re-probe. Nothing here ever throws: a
failure comes back as {:reason <why>} -- no :bytes, so the caller still
SKIPS the attachment, but it can TELL the user why instead of dropping a
perfectly valid picture in silence.
Pixels come from com.blockether/imaging, never from AWT/Java2D/ImageIO --
which is why this works identically in the native image, on every platform,
with no headless or fontconfig bootstrap. Tests do the opposite on purpose:
independent-decoder-test re-reads every payload this namespace emits with
javax.imageio, so "it decodes" is never just the encoder agreeing with
itself -- the provider's decoder is a third implementation again.
Container conversion for attachment payloads -- NOT optimization.
vis stores and replays the attachment PICTURE verbatim: whatever the user
dropped is what is stored and what the provider sees, pixel for pixel (the
stored ENCODING may be shrunk losslessly -- [[compact]]). The one thing that
cannot be passed through is a container no vision wire accepts (`attachments/
provider-image-media-types`): a BMP is a perfectly good image and an SVG is
a perfectly good figure, and both are a hard 400 on every provider. Since an
attachment REPLAYS on every later turn, one such row kills the whole session.
So this namespace does exactly one thing: turn bytes a provider REFUSES into
the same picture in a container it ACCEPTS.
* raster (BMP, TIFF, or anything else the decoder reads) -> PNG, 1:1
* vector (`.svg` / gzipped `.svgz`) -> rendered PNG
* video (`.mp4` / `.mov`) -> animated GIF
Everything a renderer can answer is ASKED, not re-implemented.
`com.blockether/imaging` (Rust `image` + resvg over FFM) decodes, sniffs,
gunzips `.svgz`, resolves `width="100%"`/`cm`/`viewBox` letterboxing, and
rasterizes. It also owns the two BROWSER repairs resvg alone refuses, behind
`imaging/svg-canvas`:
* a zero or negative declared size -- resvg refuses the document outright
("SVG has an invalid size"); browsers fall back to the `viewBox`.
* a document that declares NO size -- resvg's bounds run from the ORIGIN
(a figure at x=50 gains a 50px margin) and collapse to a bare 100x100
when content sits at negative coordinates, so the canvas is framed by the
ink actually painted.
Both used to be hand-written Clojure HERE. They are Rust in the library now --
one implementation for every caller, and vis only asks for the answer.
`renderer-delegation-test` still pins that answer document by document, so a
change in resvg or in the repair shows up as a vis test failure.
Conversion is 1:1 and never re-compresses what the wire already takes: no
quality knob, no size heuristics on the way through. There are exactly two
exceptions, both narrow and both named: [[fit-within]], which the send gate
calls ONLY for a payload over the wire's byte cap that would otherwise be
DROPPED, and [[compact]], which the storage rail calls on the way into the
`session_attachment` BLOB and which is LOSSLESS -- same picture, fewer bytes,
verified by a re-probe. Nothing here ever throws: a
failure comes back as `{:reason <why>}` -- no `:bytes`, so the caller still
SKIPS the attachment, but it can TELL the user why instead of dropping a
perfectly valid picture in silence.
Pixels come from `com.blockether/imaging`, never from AWT/Java2D/ImageIO --
which is why this works identically in the native image, on every platform,
with no headless or fontconfig bootstrap. Tests do the opposite on purpose:
`independent-decoder-test` re-reads every payload this namespace emits with
`javax.imageio`, so "it decodes" is never just the encoder agreeing with
itself -- the provider's decoder is a third implementation again.Kill switch -- bind false to make conversion unavailable (what a build with no imaging cdylib looks like).
Kill switch -- bind false to make conversion unavailable (what a build with no imaging cdylib looks like).
(compact data)Losslessly re-compress an image payload on its way INTO the store: oxipng's
filter/colour-type/palette search for PNG, jpegtran-style marker stripping
for JPEG, gifsicle's differencing re-encoder for GIF -- imaging/optimize
with NO options, which that library documents as pixel-preserving (only
:lossy/:quality/:format/:max-* license a re-encode).
The second exception to "never re-compress", and a narrower one than
fit-within: the PICTURE is untouched, only its encoding is. A matplotlib
or screenshot PNG comes out of a speed-tuned encoder and routinely costs
several times the bytes it needs -- bytes that are then paid in the
session_attachment BLOB, in every /poll + SSE replay of that row, and in
every later turn the attachment replays. What the model and the human see is
bit-identical, which is what the verbatim contract protects.
Belt and braces, because a silent quality drop here would be invisible: the result is taken ONLY when it is strictly smaller AND still probes as the same format at the same dimensions and frame count ([[same-picture?]]).
Never bigger than data, never nil, never throws.
Losslessly re-compress an image payload on its way INTO the store: oxipng's filter/colour-type/palette search for PNG, jpegtran-style marker stripping for JPEG, gifsicle's differencing re-encoder for GIF -- `imaging/optimize` with NO options, which that library documents as pixel-preserving (only `:lossy`/`:quality`/`:format`/`:max-*` license a re-encode). The second exception to "never re-compress", and a narrower one than [[fit-within]]: the PICTURE is untouched, only its encoding is. A matplotlib or screenshot PNG comes out of a speed-tuned encoder and routinely costs several times the bytes it needs -- bytes that are then paid in the `session_attachment` BLOB, in every `/poll` + SSE replay of that row, and in every later turn the attachment replays. What the model and the human see is bit-identical, which is what the verbatim contract protects. Belt and braces, because a silent quality drop here would be invisible: the result is taken ONLY when it is strictly smaller AND still probes as the same format at the same dimensions and frame count ([[same-picture?]]). Never bigger than `data`, never nil, never throws.
(fit-within data max-bytes)Squeeze an already wire-safe payload under max-bytes with the imaging
library's REAL optimisers: oxipng's filter/colour-type/palette search for
PNG, jpegtran-style lossless metadata stripping for JPEG, gifsicle's
differencing re-encoder for GIF -- and, only if the lossless pass still does
not fit, that library's own bounded ladder (quality/palette, then downscale).
This is the ONE place a byte SIZE is allowed to matter here, and it earns the exception: the alternative is not "keep the original", it is DROP -- the picture is refused and the model is told a file it can see on disk cannot be shown ([[com.blockether.vis.internal.attachments/wire-verdict]]). A payload that already fits is returned ITSELF, so the verbatim-bytes contract holds for every attachment the wire can take as it stands.
Never bigger than data, never nil, never throws.
Squeeze an already wire-safe payload under `max-bytes` with the imaging library's REAL optimisers: oxipng's filter/colour-type/palette search for PNG, jpegtran-style lossless metadata stripping for JPEG, gifsicle's differencing re-encoder for GIF -- and, only if the lossless pass still does not fit, that library's own bounded ladder (quality/palette, then downscale). This is the ONE place a byte SIZE is allowed to matter here, and it earns the exception: the alternative is not "keep the original", it is DROP -- the picture is refused and the model is told a file it can see on disk cannot be shown ([[com.blockether.vis.internal.attachments/wire-verdict]]). A payload that already fits is returned ITSELF, so the verbatim-bytes contract holds for every attachment the wire can take as it stands. Never bigger than `data`, never nil, never throws.
(rasterize-svg data)(rasterize-svg data {:keys [max-dimension]})Render an SVG (.svg or gzipped .svgz) into a PNG every vision wire
accepts. Returns {:bytes :media-type :size :original-size :width :height :original-width :original-height}, {:reason <why>} when the document does
not parse or renders to nothing, or nil when conversion is disabled. Never
throws.
The ASPECT RATIO is the whole game: a squashed chart is a wrong chart, and
the model cannot tell the difference. The raster size is the document's own
resolved size, and the only scaling that ever happens is the
svg-max-raster-dimension ceiling, one factor applied to both edges.
The canvas is filled WHITE first: SVG figures routinely leave the background transparent, and a transparent PNG of black axes is what a viewer -- or a model -- sees as an empty image.
Render an SVG (`.svg` or gzipped `.svgz`) into a PNG every vision wire
accepts. Returns `{:bytes :media-type :size :original-size :width :height
:original-width :original-height}`, `{:reason <why>}` when the document does
not parse or renders to nothing, or nil when conversion is disabled. Never
throws.
The ASPECT RATIO is the whole game: a squashed chart is a wrong chart, and
the model cannot tell the difference. The raster size is the document's own
resolved size, and the only scaling that ever happens is the
[[svg-max-raster-dimension]] ceiling, one factor applied to both edges.
The canvas is filled WHITE first: SVG figures routinely leave the background
transparent, and a transparent PNG of black axes is what a viewer -- or a
model -- sees as an empty image.Hard ceiling on the rendered long edge. A vector document can declare any size at all, and a 40000px canvas is an OOM, not a picture.
Hard ceiling on the rendered long edge. A vector document can declare any size at all, and a 40000px canvas is an OOM, not a picture.
Vector containers vis RASTERIZES on the way in instead of refusing. No
provider reads SVG -- it is markup, not pixels -- but unlike a HEIC the
pixels are recoverable: the document is rendered here, so a matplotlib
savefig(format="svg") or a dropped icon still reaches the model.
Vector containers vis RASTERIZES on the way in instead of refusing. No provider reads SVG -- it is markup, not pixels -- but unlike a HEIC the pixels are recoverable: the document is rendered here, so a matplotlib `savefig(format="svg")` or a dropped icon still reaches the model.
(to-provider-safe data media-type)Re-container data into something every vision wire accepts (PNG) -- and
never hand back bytes the wire would refuse. Payloads whose container is
already wire-safe come back UNTOUCHED, byte for byte, but only after
[[verified-raster]] has decoded them: this is a VALIDITY fix, never a size
one, and an undecodable payload is exactly the thing it must catch. Returns
the same map as rasterize-svg; {:reason <why>} when the payload can be
turned into pixels by neither the decoder nor the SVG renderer (a corrupt
raster, HEIC, AVIF, an image the decoder refuses as too large); nil when
conversion is DISABLED -- the caller skips it either way, but a reason is
something it can SHOW.
Re-container `data` into something every vision wire accepts (PNG) -- and
never hand back bytes the wire would refuse. Payloads whose container is
already wire-safe come back UNTOUCHED, byte for byte, but only after
[[verified-raster]] has decoded them: this is a VALIDITY fix, never a size
one, and an undecodable payload is exactly the thing it must catch. Returns
the same map as [[rasterize-svg]]; `{:reason <why>}` when the payload can be
turned into pixels by neither the decoder nor the SVG renderer (a corrupt
raster, HEIC, AVIF, an image the decoder refuses as too large); nil when
conversion is DISABLED -- the caller skips it either way, but a reason is
something it can SHOW.(video->wire-gif data)(video->wire-gif data opts)Turn a video container (MP4 / QuickTime) into an animated GIF every vision
wire accepts, sampled across the clip's whole length: at most
video-gif-max-frames evenly strided frames, long edge
video-gif-max-dimension.
A clip is the one attachment NO provider takes in any form, so the choice is not "convert or pass through", it is this or a blind turn -- the model gets the motion at a legible size instead of nothing at all. Sampling is strided rather than truncated on purpose: the first 24 frames of a screen recording are its first second, which is exactly the part that shows nothing.
Returns the same map as rasterize-svg plus :frames; {:reason <why>}
when the clip cannot be decoded (an HEVC/AV1/VP9 track this build does not
decode, a corrupt container); nil when conversion is DISABLED.
Turn a video container (MP4 / QuickTime) into an animated GIF every vision
wire accepts, sampled across the clip's whole length: at most
[[video-gif-max-frames]] evenly strided frames, long edge
[[video-gif-max-dimension]].
A clip is the one attachment NO provider takes in any form, so the choice is
not "convert or pass through", it is this or a blind turn -- the model gets
the motion at a legible size instead of nothing at all. Sampling is strided
rather than truncated on purpose: the first 24 frames of a screen recording
are its first second, which is exactly the part that shows nothing.
Returns the same map as [[rasterize-svg]] plus `:frames`; `{:reason <why>}`
when the clip cannot be decoded (an HEVC/AV1/VP9 track this build does not
decode, a corrupt container); nil when conversion is DISABLED.Playback rate of the produced GIF. Slow on purpose: the frames are strided samples of the WHOLE clip, not consecutive ones.
Playback rate of the produced GIF. Slow on purpose: the frames are strided samples of the WHOLE clip, not consecutive ones.
Long-edge ceiling for the GIF a clip becomes. A model reads a contact sheet,
not a film: 320px keeps a UI flow or a crash repro legible while leaving
video-gif-max-frames frames comfortably inside the per-image cap.
Long-edge ceiling for the GIF a clip becomes. A model reads a contact sheet, not a film: 320px keeps a UI flow or a crash repro legible while leaving [[video-gif-max-frames]] frames comfortably inside the per-image cap.
Frames kept from a clip. Enough to read motion, few enough that a 30s screen recording stays a few hundred KB.
Frames kept from a clip. Enough to read motion, few enough that a 30s screen recording stays a few hundred KB.
Containers no vision wire needs CONVERTED (mirrors
attachments/provider-image-media-types). Kept here so to-provider-safe
can hand these back byte-identical instead of re-encoding a payload that was
already fine -- byte-identical, but never UNCHECKED: the bytes are still
decoded once to prove they are pixels (see [[verified-raster]]).
Containers no vision wire needs CONVERTED (mirrors `attachments/provider-image-media-types`). Kept here so [[to-provider-safe]] can hand these back byte-identical instead of re-encoding a payload that was already fine -- byte-identical, but never UNCHECKED: the bytes are still decoded once to prove they are pixels (see [[verified-raster]]).
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |