boring.core — org.replikativ/boring 0.1.7

clj

Thin Clojure surface over the Java hot path.

The dispatch loop lives in Java on purpose: the per-value type test plus byte emission is where the time goes, and crossing the Clojure/Java boundary per value costs more than it saves. Measured, the Java hot path is worth 1.2-1.7x over the equivalent pure Clojure (5.8x on strings, where bulk VarHandle writes and a single-pass ASCII encode beat per-byte aset).

Options (all optional):

:profile :clojure (default) | :interop | :archival | :canonical :float-policy :preserve-width (default) | :shortest :stringref true (default under :clojure) | false

See doc/COMPATIBILITY.md for what each profile promises about the bytes. :float-policy exists because datahike's dumps must not narrow a double to a float -- the class, not just the value, has to survive.

The four profiles answer four different questions:

:clojure round-trip fidelity within Clojure, smallest bytes. :interop can any conformant CBOR reader read this? (no extensions) :archival will two exports of the same data be identical, AND do JVM types survive? (sorted keys + fixed-width floats) :canonical do these bytes agree octet-for-octet with other canonical encoders? (RFC 8949 4.2.2, which narrows floats)

:archival and :canonical are NOT the same and cannot be: RFC determinism requires the shortest float form, which discards the Double/Float distinction. Pick by which of the two you actually need.

cljs

ClojureScript surface. Mirrors the JVM boring.core API so the conformance suite in test/boring/ runs unmodified on both platforms.

Platform differences that cannot be papered over, and are documented rather than hidden:

• JS has one number type, so 1 and 1.0 are the same value. Integers encode as CBOR integers and everything else as f64; :float-policy has no meaning here beyond bignums.
• Values past 2^53 decode to BigInt rather than silently losing precision (datahike's dump requirements); values inside it stay ordinary numbers.
• Date has millisecond resolution, so nanosecond instants do not survive.

buffer clj/s≠

(buffer w)

clj

The writer's internal buffer. Valid bytes are [0, count) where count is what encode-buffered! returned.

This array must not outlive the call, and must not cross a thread or async boundary. It is the writer's live buffer, not a copy: the next encode overwrites it in place. Handing it to a go block, a future, or an async write while the calling thread loops on the same writer corrupts the bytes mid-flight, with no exception anywhere — you get a bad blob on disk and no indication of why. This is the sharpest edge in the API and the one an async storage backend hits first.

If the bytes outlive the call, use encode-into!, which copies.

decode clj/s≠

(decode bs)

(decode bs opts)

clj

Decode the first CBOR item in bs.

Errors are ex-info carrying a :type keyword — :boring/truncated-input, :boring/invalid-utf8, :boring/bad-count, :boring/max-depth-exceeded, :boring/duplicate-map-key, :boring/unregistered-tag, and so on.

:tolerate-unknown-tags (default true) makes an unregistered tag surface as a boring.data/TaggedValue; false makes it an error, which is the closed-reader behaviour datahike's dump requirements ask for.

decode-seq clj/s≠

(decode-seq bs)

(decode-seq bs opts)

clj

Lazily decode consecutive top-level CBOR items from bs.

Memory is bounded by the item being realised, not by bs — but bs itself is already in memory. Use decode-seq-from to read a stream larger than the heap.

cljs

Lazily decode consecutive top-level CBOR items (RFC 8742 sequence).

decode-seq-from clj

(decode-seq-from in)

(decode-seq-from in opts)

Lazily decode a CBOR sequence (RFC 8742) from an InputStream, in bounded memory.

The write side has streamed since the beginning (write-to!, write-seq!) while the read side took only a byte[], so a dump larger than the heap had no symmetric read path -- decode-seq's own docstring conceded the chunking workaround. This is that workaround, done once and correctly.

Bounded memory means bounded by the LARGEST SINGLE ITEM plus the chunk size, not by the stream. That is the real limit and it is not a compromise: an item has to fit in memory to be a Clojure value at all, so streaming can only ever mean a sequence of items -- which is exactly what a datahike dump is.

The reader's hot path is untouched. Refilling happens between items, not inside u8(), so this costs nothing when decoding from a byte array.

:chunk-size (default 64 KiB) is how much is pulled from the stream at a time. The caller owns the stream and should close it.

decode-with clj/s≠

(decode-with r bs)

(decode-with r bs opts)

clj

Decode using a reusable Reader.

With opts, every option is re-applied on this call. Without them the reader keeps whatever it was last configured with, which is what makes the two-arity form fast and also what makes it a state-leak hazard across tenants -- pass opts unless the reader is yours alone.

cljs

Decode using a reusable Reader. With opts, every option is re-applied on this call; without them the reader keeps its previous configuration.

encode clj/s≠

(encode v)

(encode v opts)

clj

Encode v to a fresh byte[].

encode-buffered! clj/s≠

(encode-buffered! w v)

(encode-buffered! w v opts)

clj

Encode v into w and return the byte COUNT, without copying anything out.

The bytes live in the writer's own buffer — reach them with buffer or hand them to a stream with write-to!. With a reused writer this makes the encode loop allocation-free apart from the buffer's own growth, which is the property hako reaches for with off-heap segments; on-heap gets there too as long as nobody insists on a freshly-allocated byte[] per message.

The buffer is overwritten by the next encode. Do not retain it.

cljs

Encode into w and return the byte count, copying nothing out. Reach the bytes with buffer; they are overwritten by the next encode.

encode-into! clj/s≠

(encode-into! w v)

(encode-into! w v opts)

clj

Encode v using the reusable writer w, returning a byte[]. Reusing w avoids reallocating the buffer and the stringref table between calls.

Still allocates the returned array. For a fully allocation-free loop use encode-buffered! with buffer/write-to!.

hasch-integration? clj

True if the optional hasch integration is active.

reader clj/s≠

(reader bs)

(reader bs opts)

clj

A reusable Reader. opts are the same map decode takes.

The one-arity form left every option at its default, and decode-with had no opts arity at all, so the advertised reusable-reader path could not be given a registry, a depth cap or a date type without reaching into the Java fields. An API that is documented as the fast path has to accept the same configuration as the slow one.

cljs

A reusable Reader. opts are the same map decode takes.

register-record clj/s≠

(register-record reg wire-name map-ctor)

clj

Teach the reader how to rebuild a record from its field map, keyed by the name it carries on the wire. Returns a NEW registry.

This is the portable form — it has the same signature and the same threading idiom on the JVM and on ClojureScript, so registration code can live in a .cljc file:

(defrecord Point [x y])

(def registry (-> (boring/tag-registry) (boring/register-record "my.ns.Point" map->Point)))

The wire name is boring.data/record-type-name of an instance — on the JVM the class name, and ClojureScript munges its own name to match, so a record written on either platform reads on the other under one registration.

Writing needs no registration: a record always encodes with its own type name, so the type is never silently flattened to a map. Without a registration a record decodes to a boring.data/UnknownRecord carrying the same name and fields, which re-encodes to identical bytes.

Security: the reader looks wire-name up in this registry. There is no Class.forName path, so a hostile document cannot cause an arbitrary class to be instantiated. See register-record-class for the JVM-only reflective convenience.

cljs

Teach a registry how to rebuild a record from its field map, keyed by the name it carries on the wire. Returns the registry.

Same signature and same threading idiom as the JVM's register-record, so registration code can live in a .cljc file:

(defrecord Point [x y])

(def registry (-> (boring/tag-registry) (boring/register-record "my.ns.Point" map->Point)))

The constructor must be passed explicitly here: advanced compilation minifies constructor names, so there is nothing to reflect on. record-type-name munges ClojureScript's own name to match the JVM class name, so one registration serves data written on either platform.

Both platforms are immutable, so reg is left untouched and the return value must be threaded. There is no process-global registry on either side.

register-record-class clj

(register-record-class reg cls)

(register-record-class reg cls wire-name)

JVM-only convenience: derive both the wire name and the map->Name constructor from cls by reflection. Returns a NEW registry.

(defrecord Point [x y])

(def registry (-> (boring/tag-registry) (boring/register-record-class Point)))

Equivalent to register-record with the class name and map factory looked up for you. Not available on ClojureScript, where advanced compilation minifies constructor names — use register-record there (and in .cljc).

Pass wire-name to override the name used on the wire, in both directions.

register-records clj/s

(register-records reg ctors)

Register many record constructors at once, from a map of wire name -> map->Record. Returns a NEW registry.

Equivalent to threading register-record over the map, but one operation rather than N. On the JVM that matters: a registry copies its whole backing map per registration, which is the right trade for one built at startup and the wrong one for a caller that derives a registry per operation. konserve's serializer protocol hands you handlers per read, so the natural fold cost N map copies per read and overtook fressian past ~20 handlers.

Keys are stred, so a map keyed by symbols -- incognito's shape -- works directly.

Still memoise if you derive a registry per operation; this only makes the un-memoised path O(1) copies instead of O(N).

register-tag clj/s≠

clj

(register-tag reg tag cls write-fn read-fn)

Teach a registry about tag for values of cls. Returns a NEW registry.

write-fn : (fn [value] -> encodable) — result is written as the tag content read-fn : (fn [decoded-content] -> value)

Registering only one direction is fine; pass nil for the other. Same signature and same threading idiom on both platforms:

(def registry (-> (boring/tag-registry) (boring/register-tag 40001 java.net.URI str #(java.net.URI. %))))

cljs

(register-tag reg tag type write-fn read-fn)

Teach a registry about tag for values of type. Returns a NEW registry.

Same signature and same threading idiom as the JVM side — registries are immutable values on both platforms:

(def registry (-> (boring/tag-registry) (boring/register-tag 40001 js/URL #(.-href %) #(js/URL. %))))

tag-registry clj/s≠

(tag-registry)

clj

The empty registry. Registries are immutable values — build one with register-tag / register-record and pass it as :registry.

There is deliberately no process-global default. Independent libraries in one JVM would otherwise register into the same namespace, and two registrations of the same tag would resolve by namespace load order.

cljs

An empty registry. Shape:

{:writers {Type {:tag n :fn value->encodable}} :readers {tag-n content->value} :records {"wire.Name" map->Record}}

Pass as :registry. Security matches the JVM side: reading dispatches on a tag NUMBER or a record NAME looked up here — it never resolves a symbol or evaluates anything from the wire.

unencodable clj/s≠

(unencodable x)

clj

The default :encode-fallback placeholder: a tag-27 frame naming the type that could not be encoded, with its pr-str.

Readable by any CBOR implementation, and obviously a placeholder rather than a value that might be mistaken for the original.

cljs

The default :encode-fallback placeholder -- see the JVM core.

write-seq! clj/s≠

clj

(write-seq! w values out)

(write-seq! w values out opts)

Encode each value in values to out as consecutive top-level CBOR items. Returns the number of bytes written. Constant memory: one value at a time, through the writer's own buffer, with no intermediate array per item.

cljs

(write-seq! w values sink)

(write-seq! w values sink opts)

Encode each value as a consecutive top-level item, appending to sink, a function of one Uint8Array. Returns the total byte count.

sink receives bytes it OWNS. This used to pass .subarray of the writer's reusable buffer, which is a view rather than a copy: a sink that retained the Uint8Array -- the natural reading of "appending to sink" -- saw every retained item overwritten by the next iteration, and a sink that finished asynchronously saw whatever the buffer held by then. The buffer hazard is documented for buffer, where the caller asks for it; here it was hidden inside a higher-level API that reads as safe.

Use encode-buffered! with buffer if you want the borrowed view and will consume it synchronously.

write-to! clj

(write-to! w v out)

(write-to! w v out opts)

Encode v into w and write its bytes straight to out, with no intermediate array.

writer clj/s≠

clj

(writer)

(writer initial-size)

Create a reusable Writer. Not thread-safe; one per thread or per loop.

cljs

(writer)

(writer size)