Liking cljdoc? Tell your friends :D

Interop

boring emits standard CBOR. Any conformant decoder in any language parses its output without error — that is what "self-describing" means, and it is the whole reason for choosing CBOR.

What this document answers is the next question: once it parses, what do you actually get, and how do you turn it into something idiomatic?

Everything here is executed in CI. interop/test_read_boring.py runs on every push against a committed fixture, so if boring's wire format changes and this document stops being true, the build breaks. A snippet pasted into a markdown file rots silently; this one cannot.

The tag table

tagmeaningwhat a generic decoder gives youidiomatic form
ints, floats, strings, bytes, arrays, maps, booleans, nullthe obvious thing
0, 1date/timehandled natively by most librariestimestamp
2, 3bignumhandled nativelybig integer
4decimal fractionhandled nativelydecimal, scale preserved
25, 256stringrefhandled natively by cbor2; a compression detailtransparent
27generic object[type-name, field-map]your record type
30rationalhandled nativelyfraction
32URIa tagged stringjava.net.URI
35regular expressiona tagged stringregex (source only, no flags)
37UUIDhandled nativelyUUID
40multi-dimensional array[dims, flat-array]row-major matrix
39identifiera tagged stringkeyword or symbol
77–86RFC 8746 typed arrayoften unimplemented — raw bytesarray of numbers
258sethandled natively by cbor2set
1002duration (RFC 9581)a keyed mapjava.time.Duration
1004full-date (RFC 8943)a date stringjava.time.LocalDate
39649shaped array — boring's one extension[keys, [row-values…]]array of maps

Three rows deserve attention.

Tag 39 is where Clojure shows through. A keyword is a string with a leading :; a symbol is the same tag without one. Distinguishing them matters: if you flatten both to plain strings, :a and "a" become the same map key and you have silently merged two entries.

Tags 77–86 are registered, standard, and frequently unimplemented. Python's cbor2 hands back a raw CBORTag(79, b"…") rather than a list. This is worth handling rather than avoiding — the payload is a plain little-endian memory image, so a homogeneous numeric column costs one bulk read instead of one decode per element. It is the fastest thing boring emits, and unpacking it is one struct call.

Tag 39649 is the only thing here that is not a registered CBOR tag. See below.

Reserved tag-27 names

Tag 27 is "serialised language-independent object with type name and constructor arguments". boring uses it for your records — the type name is the record's class name — and also for a handful of types CBOR has no tag for, under names carrying a slash, which a JVM class name never does:

nameargumentwhy it is not just the bare value
clojure/sorted-mapa mapa sorted map is a CBOR map; unmarked it returns unsorted
clojure/sorted-setan arraysame, for sets
clojure/queuean arraya queue is a CBOR array; unmarked it returns a vector
clojure/with-meta[meta, value]Clojure metadata, which has nowhere else to go
clojure/chara 1-character string(= \a "a") is false in Clojure
java/periodan ISO-8601 string, e.g. "P1Y1M1D"a date amount; RFC 9581's tag 1002 carries seconds

The prefix names the runtime that owns the type. These names are frozen — they are pinned in the golden corpus, because renaming one is a silent break: a reader that does not recognise a name yields an ordinary value rather than raising.

For a foreign reader, ignoring them entirely is safe and lossy in a predictable direction — a sorted map arrives as a map, a queue as an array, a char as a one-character string. Handling clojure/with-meta is worth the four lines, since otherwise every annotated value arrives wrapped:

def tag27(name, arg):
    if name == "clojure/with-meta":
        return tag27_value(arg[1])       # or keep arg[0] if you want the meta
    if name in ("clojure/queue", "clojure/sorted-set"):
        return arg
    ...

Only clojure/with-meta changes the shape of what you get; the rest change only the type, which a language without sorted maps or characters was going to lose anyway.

The one extension

An array whose elements are all maps sharing one key set is written with the keys once:

value    [{:e 1 :a :x} {:e 2 :a :y}]

wire     39649([ [":e", ":a"],            <- the key set, once
                 [ [1, ":x"],             <- row 1, values only
                   [2, ":y"] ] ])         <- row 2

Reconstructing it is a zip, and it needs no state outside the tag — that is the property that makes it easy to support elsewhere, and the reason it was designed this way rather than as a stream-scoped structure table:

def shaped_array(payload):
    keys, rows = payload
    return [dict(zip(keys, row)) for row in rows]

It is off by default (:shapes true opts in), and a decoder that ignores it gets an inert tagged value — never a misreading. Use {:profile :interop} to guarantee no extension appears at all.

It is not yet registered with IANA, so treat the number as provisional. The registration is tracked in COMPATIBILITY.md.

Python

interop/read_boring.py is a complete reader in about 60 lines of logic. It is the file CI runs.

from read_boring import loads

value = loads(open("data.cbor", "rb").read())

cbor2 already handles tags 0–4, 25/256, 30, 37 and 258 natively. What read_boring adds:

def _tag_hook(decoder, tag):
    if tag.tag == 39:                       # identifier
        s = tag.value
        return Keyword(s) if s.startswith(":") else Symbol(s)
    if tag.tag == 27:                       # record
        type_name, fields = tag.value
        return Record(type_name, fields)
    if tag.tag == 39649:                    # shaped array
        keys, rows = tag.value
        return [dict(zip(keys, row)) for row in rows]
    if tag.tag in TYPED_ARRAYS:             # RFC 8746
        fmt, width = TYPED_ARRAYS[tag.tag]
        return list(struct.unpack(f"<{len(tag.value) // width}{fmt}", tag.value))
    return tag

def loads(data):
    return cbor2.loads(data, tag_hook=_tag_hook)

Run it yourself:

pip install cbor2
python3 interop/test_read_boring.py

The test asserts 24 values against literal expectations, including that a decimal keeps its scale — Decimal("1.50") == Decimal("1.5") is True in Python, so equality alone would not catch that loss, and the test checks as_tuple().exponent instead.

Rust

interop/rust/ is a complete reader in ~350 lines of ciborium::value::Value walking — no boring-specific crate, because there isn't one and none is needed. It reads the same committed fixture the Python reader does and checks the same values, so the two languages agree or CI says so:

cargo run --manifest-path interop/rust/Cargo.toml -- interop/fixture.cbor

Two things it has to do that a naive reader will not, and both are worth knowing before you write your own:

  • Stringref is resolved before anything else. ciborium does not implement tags 25/256, so the reader walks the raw value tree once and substitutes back-references. The table-building rule must match the writer's exactly — a string is only entered if referencing it would be shorter than repeating it — because one disagreement makes every subsequent index wrong. Use {:stringref false} if you would rather not implement it.
  • A tag-258 set is an array, and its order means nothing. This fixture emits 1, 3, 2. Comparing set contents positionally passes on some values and fails on others, which is worse than failing consistently.

Go, JavaScript

The same four hooks, against these libraries:

These are not yet executed in CI, unlike the Python reader. Treat them as guidance until they are — the shape of the problem is identical, but I would rather say plainly that they are untested than imply a coverage that does not exist.

Guaranteeing no extensions

If you are writing data for a consumer you do not control:

(boring/encode value {:profile :interop})

That disables stringref and shapes. Every byte is then a registered CBOR construct, and any conformant decoder produces the right structure without a single hook — at the cost of a larger document.

Can you improve this documentation?Edit on GitHub

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close