com.blockether.vis.ext.channel-tui.video — com.blockether/vis-channel-tui 0.1.22

MP4 (H.264) → frames, so a clip can be WATCHED inline in a graphical terminal.

The codec lives in com.blockether/imaging, NOT here: imaging/probe-video, imaging/decode-video and imaging/video->gif are the pure-Rust demuxer (re_mp4) + H.264 decoder (rust_h264) inside the same cdylib that already does every still image in vis. That means no ffmpeg on PATH, no java.desktop, nothing to install, and it survives the GraalVM native image — while jcodec stays on this jar's classpath for what it is genuinely good at, ENCODING a replayed session in channel-tui.cinema.

What is left in this namespace is the TERMINAL half: brand sniffing, sizing a clip to a cell grid, and the Kitty/iTerm2 escape sequences that actually animate it. Decoding pixels is imaging's job.

Reach for ->gif first. An animated GIF is the format the WHOLE existing stack already understands — foundation.gif, the attachment sniffer, the provider wire (image/gif is one of the four verbatim-legal vision types) and iTerm2's native inline animation — so transcoding a clip is what makes it both viewable and describable-by-the-model without touching any of that plumbing. ->gif never materialises frames on the JVM heap at all: the transcode happens inside the cdylib and only GIF bytes come back. decode-frames is the lower level for a frame-stepping Kitty player.

Frames arrive in DISPLAY order (B-frames reordered), display-cropped (H.264 pads to whole macroblocks, so 1080p codes as 1920×1088) and as straight RGBA8, because the decoder resolves all three before the bytes cross the FFI.

H.264 is the supported codec. HEVC/H.265 — what a modern iPhone records by default — AV1 and VP9 are demuxed but NOT decoded, and that is reported as a clear ex-info rather than a corrupt picture.

->gif clj

(->gif src
       {:keys [max-frames stride max-dimension fps]
        :or {max-frames default-max-frames stride 1}})

Transcode the MP4 at src into ONE animated GIF byte array, looping forever.

This is the format that makes a clip work everywhere the rest of vis already works — iTerm2 animates it inline with no player loop, foundation.gif round-trips it, and image/gif is legal on the provider vision wire, so the model can be shown a clip instead of being told about one.

The whole transcode runs inside imaging's cdylib, so the frames never touch the JVM heap. Takes decode-frames's opts; :max-dimension defaults to default-gif-max-dimension and :fps overrides the playback rate (GIF stores delays in centiseconds, so the rate is quantised to 1/100 s).

decodable? clj

(decodable? src)

True when src is an MP4 whose video codec this decoder actually supports.

decode-frames clj

(decode-frames src)

(decode-frames src
               {:keys [max-frames stride max-dimension encoding]
                :or {max-frames default-max-frames stride 1}})

Decode src (a File or path to an H.264 MP4) into

{:width :height :fps :frames [{:index :timestamp-s :width :height :rgba} …]}

where :rgba is one frame's straight RGBA8 canvas and :index its position in the ORIGINAL stream (so a strided decode still reports true timing). :width/:height describe the returned frames, after any scaling.

opts: :max-frames stop after this many KEPT frames (default default-max-frames) :stride keep every Nth frame — 2 halves the frame rate (default 1) :max-dimension downscale each frame so its long edge fits this :encoding :rgba (default) or :png

:encoding :png makes the cdylib encode each frame itself and hands back :png bytes in place of :rgba. That is what both terminal protocols transmit anyway, and it is strictly cheaper: frames cross the FFI boundary base64'd, so a 120-frame 720p clip moves ~9 MB of PNG instead of ~140 MB of RGBA — faster than decoding raw and re-encoding here, and playback then holds megabytes instead of hundreds of them.

Throws ex-info with :reason :not-mp4 / :unsupported-codec rather than returning something subtly wrong.

default-gif-max-dimension clj

Long-edge ceiling ->gif scales down to. A GIF is palette-quantized and LZW-packed per frame; full-resolution video turns into tens of megabytes that no vision wire would accept anyway.

default-max-frames clj

Frames decode-frames will hold in memory before it stops. Every frame is a full uncompressed RGBA canvas, so a 1280×720 clip costs ~3.7 MB per frame — 240 of them is already ~880 MB unscaled. Callers that want a long clip pass :max-dimension (and usually :stride) rather than raising this.

frame->png clj

(frame->png {:keys [width height rgba png]})

ONE decoded frame → PNG bytes, via imaging (no java.desktop).

A frame decoded with :encoding :png already CARRIES its PNG, so this is free for the playback path; an :rgba frame is encoded here.

media-type clj

(media-type src)

The video/… media type for src, or nil when it is not a clip we can open. The NAME only picks the label (.mov → video/quicktime); the verdict is always the magic-byte sniff, so a mislabelled file cannot fool a caller.

mp4? clj

(mp4? data)

True when data begins with an ISO base-media (ftyp) box whose major brand is a VIDEO brand — .mp4, .m4v and .mov all land here. HEIF/AVIF still images share the container and are rejected by the fork's brand table.

play! clj

(play!
  src
  {:keys [out loops cols rows max-dimension encoding] :or {loops 1} :as opts})

Play the MP4 at src inline in a graphical terminal — the whole point of this namespace.

Decodes, then paces the frames against a monotonic clock so playback keeps real time instead of drifting by however long each PNG encode took. Options are decode-frames's plus:

:out OutputStream to write to (default System/out) :cols :rows cell box to fit the picture into (default 80 cols, free rows) :protocol force :kitty / :iterm2 instead of sniffing the terminal :loops how many times to play (default 1)

Returns {:frames :protocol :cols :rows}, or nil when the terminal cannot draw inline images (a non-graphical TERM — nothing is written in that case).

playback-sequences clj

(playback-sequences {:keys [width height fps frames] :as decoded}
                    {:keys [protocol cols rows] :as _opts})

Turn a decode-frames result into the terminal byte stream that PLAYS it.

Returns {:protocol :cols :rows :frame-count :frames [{:index :delay-ms :escape}]}, where :escape is the full sequence drawing that one frame at the saved cursor position and :delay-ms is how long to hold it. Concatenating every :escape with the right pauses between them IS the playback — see play!.

Takes the DECODED map, not a file: (playback-sequences (decode-frames src) opts). Handing it a File/path throws rather than quietly returning zero frames.

protocol defaults to whatever the host terminal speaks (terminal-image/images-protocol); pass it explicitly to render for a specific terminal (what the tests do). Returns nil when neither the given nor the detected protocol can draw an image at all.

poster clj

(poster src)

(poster src {:keys [max-dimension]})

{:png <bytes> :width :height} for ONE frame of src — the STILL picture a terminal, a paste probe or a transcript fence shows to stand for a clip.

:max-frames 1 stops the decoder at the first picture, :max-dimension makes it scale during conversion and :encoding :png has the cdylib hand the PNG straight back, so this costs a single keyframe at the size you will actually draw (~0.2 s for 1080p, ~0.1 s for 720p) no matter how long the clip runs. Decoding the timeline just to keep frame 0 would cost seconds and hundreds of megabytes of RGBA.

nil when src holds no decodable picture.

probe clj

(probe src)

Cheap {:codec :codec-string :width :height :frames :duration-s :fps :has-audio :is-decodable} for src (a File or path), read from the MP4 index WITHOUT decoding a single picture. nil when src is not a readable MP4 or carries no video track.

:codec is a lowercase keyword (:h264, :hevc, …); only :h264 can be decoded — see decodable?. :codec-string keeps the container's precise RFC 6381 name ("avc1.420028").

video-file? clj

(video-file? src)

True when src is an existing readable file whose FIRST BYTES are a video ftyp box — 12 bytes read, no decode and no index walk. This is the cheap gate every still-image surface uses to ask "is this thing a clip?" before spending a poster frame on it, so it must stay allocation-free and total; the fork's isVideoFile is exactly that.