Changelog

All notable, user-visible changes to konserve are documented here.

Unreleased

Added

• Monotonic write stamps — :last-write metadata is now issued by a process-global monotone clock (utils/now = max(wall-clock, previous-stamp)): stamps never go backwards under wall-clock retreat (NTP step-backs, VM suspend/resume, manual clock sets) — the stamp holds at its high-water mark until real time catches up. This makes konserve.gc/sweep!'s safety argument hold by construction: an object written under a collector's guard can no longer be stamped before the cutoff that protects it and be deleted while live. Deliberately non-strict (no +1 per stamp): a strict clock would run ahead of physical time above 1000 writes/second and stall collection after a restart following a bulk import; same-millisecond ties remain possible and are fail-safe (the sweep spares equality — garbage retained one cycle, never a live deletion). Stamps still read as wall time (Date); external collectors must obtain their cutoffs from utils/now/utils/monotonic-now-ms (the same source) rather than raw clock reads. Single-process only, as before.

• PReadMissSafe marker protocol (konserve.impl.storage-layout). A backing store implements it to declare that a read of an absent key is side-effect-free and reports the miss cleanly — its -read-header throws (store-key-not-found-ex store-key) on an absent key, with no side effect. When a backing declares it, io-operation learns existence from the read itself and skips the separate -blob-exists? probe, removing a redundant round-trip (an S3 HEAD before the GET). The default filestore deliberately does not implement it — its -create-blob opens with CREATE and would materialise an empty blob on a probe-free missing read — so filestore behaviour is unchanged. New helpers: store-key-not-found-ex, store-key-not-found?, store-key-not-found.

• dissoc opt :ignore-existence?. dissoc normally probes with -blob-exists? so it can return whether the key existed (true) or was absent (false) — konserve's contract, enforced by the compliance suite. A caller that does not need that boolean (e.g. a GC bulk sweep) can pass {:ignore-existence? true} to skip the probe on a PReadMissSafe backing (whose delete is idempotent), returning true. On non-miss-safe backings the hint is ignored and the probe stays.

• The IndexedDB backend implements PReadMissSafe. A browser read was two IndexedDB transactions — .getKey (the -blob-exists? probe) then .get — and is now a single .get (read-modify-write ops drop their .getKey too). Its -create-blob is side-effect-free and read-blob now signals store-key-not-found-ex on an absent key. (dissoc's single-key fast path also honours :ignore-existence?; the multi-key GC delete path is a separate follow-up.)

• :BoringSerializer (serializer byte 3) — boring, an RFC 8949 CBOR codec that runs on the JVM and ClojureScript from one implementation. Unlike :CBORSerializer (byte 2, clj-cbor) it accepts read handlers rather than throwing on them, and unlike :FressianSerializer it is not JVM-only. Because the payload is standard CBOR, a store written by Clojure can be read by any language with a CBOR library: interop/read_konserve_blob.py is the whole format in one file, and konserve.interop-python-test runs it against a real blob so it cannot drift from what konserve writes.

• :zstd compressor (compressor byte 2), via the optional dependency com.github.luben/zstd-jni. Loaded reflectively so the native binaries stay out of every user's dependency graph; when absent, byte 2 resolves to a compressor that throws an actionable message instead of the namespace failing to load. On one 512-datom blob zstd-3 was 23x faster than LZ4-HC and about half the size, which is why :lz4 stays the fast compressor rather than being switched to the high one.

Fixed

• Compression combined with encryption never worked. The read path nested the wrappers the opposite way round from the write path, so it tried to decompress ciphertext: zstd + aes failed with "Unknown frame descriptor", lz4 + aes with "Stream unsupported". Both defaults are null, and null is identity, so the order was invisible in every configuration the tests covered — while the README documents compression and encryption configured together. A new konserve.compressor-encryptor-matrix-test walks every compressor × encryptor × serializer combination through a real store, and asserts that a compressed+encrypted blob is smaller than an encrypted-only one, which pins the order rather than just the round trip.
• The LZ4 compressor truncated its own frames. It called .flush rather than .close, and LZ4FrameOutputStream.flush does not write the frame's EndMark. Fressian never noticed, because it stops reading at the end of a value; the CBOR serializer reads to EOF and hit the truncation with "Stream ended prematurely". A compressor bug that only one serializer could see, which is why the matrix runs over both.
• LZ4 was broken on every GraalVM JDK, not just native images. native-image-build? tested only whether org.graalvm.nativeimage.ImageInfo was on the classpath — and that class ships with every GraalVM JDK, so an ordinary JVM run on GraalVM took the native-image path: NPE on write, throw on read. It now asks ImageInfo/inImageCode, which is the question that was meant.
• A :frontend-only tiered store no longer deletes the shared backend. -delete-store :tiered deleted the backend unconditionally. Under :write-policy :frontend-only the store is a read-through cache over a backend that another peer OWNS and that this one must never write — deleting is the most destructive write there is, so a cache peer calling delete-store (or Datahike's delete-database) would take the authoritative data with it. It now deletes only its own cache; under every other policy the store owns its backend and both tiers go. This was latent while tiered delete silently did nothing (see below) — fixing the missing await made it reachable.
• Node file backend: delete-store-async was broken in three ways, and never ran. Wiring -delete-store :file to the async variant (above) exposed it. iofs/arm-r yields [?err] — a vector — but it was bound as a bare ?err, so the success value [nil] was truthy and the function always took the error branch: it returned [nil] as if it were an error and never reached the fsync at all. Once that was fixed, two more surfaced: it fsynced base — the directory arm-r had just deleted — where the sync twin correctly fsyncs the parent; and sync-base-async called .force on the result of open-async-file-channel without checking whether it was an Error. It now returns nil on success and the error on failure, matching the sync delete-store.
• delete-store now honours :sync? — and :tiered actually deletes. -delete-store was the one store method that ignored its opts: :memory and :file (JVM and Node) returned a plain value whatever :sync? said, so an async caller could not await the deletion — and delete-store defaults to {:sync? false}, so async is the common path. Worse, :tiered called (delete-store backend-config) with no opts — the async default — and then dropped the returned channel, so deleting a tiered store over an async backend (e.g. S3) removed nothing at all, silently, with any error swallowed into a channel nobody read. All four implementations now follow the same contract every other store method obeys: a value under {:sync? true}, otherwise a channel that delivers when the deletion is complete. The Node file backend now also uses its existing non-blocking delete-store-async on the async path. The contract is documented on the -delete-store multimethod and pinned by tests (previously memory-store-delete asserted the broken behaviour).

Changed

• Probe-elision now covers non-overwrite writes, not only reads. On a PReadMissSafe backing, update-in / update / nested assoc-in / bassoc read the old value read-first (an absent key → a fresh write) instead of a HEAD probe followed by the read. A read-modify-write on an existing key drops from HEAD + GET + PUT to GET + PUT. Pure reads (get / bget / get-meta) were already a single GET on a miss-safe backing.
• konserve.gc/sweep! passes :ignore-existence? on its single-key delete fallback, so GC on a miss-safe store deletes each dead key without a per-key HEAD probe (the batch multi-dissoc path was already probe-free).