com.blockether.vis.internal.attachment-storage — com.blockether/vis 0.1.12

Attachment storage-offload rail: a registry of storage BACKENDS plus the pure OFFLOAD DECISION that routes one attachment's payload either INLINE (bytes in the session_attachment.bytes BLOB) or EXTERNAL (bytes handed to a backend, which returns a storage_uri -- scheme://... -- kept in the row instead).

Zero SQL, zero schema change: the V4 session_attachment table already carries a nullable storage_uri with an exactly-one(bytes, storage_uri) CHECK. This namespace only decides WHICH of the two a given attachment takes, PUTs/GETs the external bytes through the scheme-dispatched backend, and hydrates a read-back envelope's :base64 from its :storage-uri on demand.

The decision is a PURE predicate hot? AND size (see default-offload?): an image replays to a vision model every turn its iteration stays live, so it is HOT -- kept inline even when large; a non-image artifact (PDF/CSV/wav/ download) is fetched at most once by a human, so it is COLD -- a good offload candidate past a size floor. A backend may override the predicate wholesale via :storage/offload?.

Precedence, engine-owned so the loop never learns a storage dialect:

1. active backend's :storage/offload? (the backend knows its own cost)
2. else default-offload? (engine default policy)
3. no active backend -> always inline (zero regression)

active-backend clj

(active-backend)

Highest-priority registered backend, or nil when none. Nil means every attachment stays inline -- the zero-backend behaviour, unchanged.

attachment-backend clj

(attachment-backend backend)

Validate + normalize a storage backend descriptor. Returns the descriptor with :storage/priority coerced to a long (default 0).

Required keys: :storage/id keyword identity, e.g. :file :storage/scheme the URI scheme string it owns, e.g. "file" -- the GET dispatch key (a storage_uri is "<scheme>://...") :storage/put-fn ({:bytes ^bytes :media-type :kind :filename :id} -> uri-string) writes the bytes, returns the handle :storage/get-fn (uri-string -> ^bytes | nil) reads the bytes back Optional: :storage/offload? ({:size :media-type :kind :tool-call-id} -> boolean) overrides default-offload? when present :storage/priority long; higher wins when several are registered (default 0)

default-offload-floor-bytes clj

Size floor below which the engine default keeps an artifact inline (256 KiB). Small payloads never earn an external round-trip.

default-offload? clj

(default-offload? {:keys [size media-type]})

Engine default offload predicate: offload a COLD artifact past the size floor. Images are HOT (replayed to a vision model every turn their iteration is live) so they NEVER offload by default, regardless of size -- otherwise each turn would re-fetch the bytes. Pure; safe to call on any attachment map.

deregister-backend! clj

(deregister-backend! id)

file-backend clj

(file-backend {:keys [dir id priority offload?] :or {id :file priority 0}})

Reference storage backend that offloads bytes to a local DIRECTORY, one file per artifact, addressed by a file://<absolute-path> URI. A genuinely useful default -- offload big COLD artifacts to disk instead of bloating the DB -- AND the reference implementation the registry contract is exercised against.

NOT auto-registered: an operator or extension opts in with (register-backend! (file-backend {:dir "/var/vis/attachments"})).

Opts: :dir (required) target directory (created if absent); :id backend id (default :file); :priority (default 0); :offload? an optional predicate override (nil -> the engine default-offload?). Each PUT mints its own opaque filename, so the storage key is independent of the DB row id.

hydrate clj

(hydrate att)

Read-back envelope -> same envelope with :base64 populated. A row stored INLINE already carries :base64 (returned untouched). A row stored EXTERNAL carries only :storage-uri, so its bytes are fetched via resolve-bytes and base64-encoded into :base64. Left alone when it already has :base64, has no :storage-uri, or the fetch fails (the caller still sees :storage-uri).

hydrate-all clj

(hydrate-all atts)

Map hydrate over a read-back attachment seq (nil-safe).

offload-attachment clj

(offload-attachment att)

Route ONE store-bound attachment map. An inline candidate carries :base64. When a backend is active AND offload? says so, decode + PUT the bytes and return the map with :storage-uri set (and :size fixed to the real byte count) and :base64 DROPPED; otherwise the map is returned unchanged. NEVER throws -- a decode/PUT failure or a non-string URI falls back to inline.

offload-attachments clj

(offload-attachments atts)

Optimize, then map offload-attachment over a store-bound attachment seq. The value handed to db-store-iteration! / db-store-session-turn! :attachments; the in-memory replay copy stays inline (offload only the persisted path).

Images pass through image-optimize/optimize-attachments FIRST: they never offload (hot), so shrinking the payload is the only lever on what an image costs on disk and on every replay to the model.

offload? clj

(offload? backend att)

Decide inline (false) vs external (true) for ONE attachment, given backend (typically active-backend). Precedence: the backend's :storage/offload? when present, else default-offload?. Nil backend -> false. Never throws -- a predicate that blows up falls back to inline.

register-backend! clj

(register-backend! backend)

Register a storage backend. Idempotent by :storage/id. Returns the normalized descriptor.

registered-backends clj

(registered-backends)

Registered storage backends ordered by descending priority (id breaks ties).

resolve-bytes clj

(resolve-bytes storage-uri)

Fetch the raw bytes for an external storage-uri by dispatching on its scheme to the owning backend's :storage/get-fn. nil when no backend owns the scheme, or the GET fails / returns nil.

scheme-of clj

(scheme-of uri)

URI scheme of a storage-uri ("<scheme>://..."), or nil.