clj-protobuf

The protobuf runtime for code generated by protoc-gen-clojure: records in, protoc's bytes out. Editions supported through 2024.

;; deps.edn
com.github.bpalermo/clj-protobuf {:mvn/version "0.1.6"}

What it is

protoc-gen-clojure emits a defrecord per message plus X->proto / proto->X conversion fns; this library is everything those fns call — the descriptor machinery, the field codec, and byte-level encode/decode:

(require '[clj-protobuf.core :as pb]
         '[acme.greeter.greeter :as g])   ; generated

(-> (g/map->HelloRequest {:name "world" :repeat-count 2})
    g/HelloRequest->proto
    pb/encode)                            ; -> bytes, byte-identical to protoc's

(->> bytes
     (pb/decode g/HelloRequest-prototype)
     g/proto->HelloRequest)               ; -> record; absent fields are nil

Records and plain maps are interchangeable everywhere a message value goes.

Semantics worth knowing

• nil means absent, both directions. Setting nil sets nothing; reading an unset explicit-presence field returns nil. Fields with IMPLICIT presence (editions) and proto3 no-label scalars have no absence — they read back as their value, default included.
• The proto field name is the authority. Kebab-cased keys are derived from it (camelCaseField → :camel-case-field); STYLE_LEGACY files work because nothing ever converts backwards.
• Enums are keywords of the exact proto value name (:COLOR_RED), lossless in both directions; numbers, strings and EnumValueDescriptors are accepted on the way in.
• Editions are the descriptor's problem. The generated file embeds its FileDescriptorProto; protobuf-java resolves presence, DELIMITED encoding and utf8 validation when the descriptor builds. This library carries no edition-specific code, which is why new editions need a protobuf-java bump and nothing else.
• Unknown fields survive on parsed Messages but are necessarily dropped by a record round trip; pb/unknown-fields inspects them when it matters.

Performance

Generated code carries a Java-class hint per message. When the matching java_proto_library classes are on the classpath the prototypes silently switch from DynamicMessage to the generated classes, with byte-for-byte identical output (the byte-identity suite proves both arms against protoc's own Java backend).

Measured with bazel run //bench:run -- quick (criterium; JDK 21, Linux x86_64; mean latency / allocated bytes per op; full Clojure-data-to-bytes pipelines). java is protoc's generated builders driven directly; jsonista and data.json carry the same value as JSON:

Encode (Clojure data → bytes)

shape	java	hinted	dynamic	jsonista	data.json
tiny	69 ns / 96 B	135 ns / 136 B	406 ns / 448 B	280 ns / 608 B	884 ns / 624 B
flat	495 ns / 400 B	860 ns / 488 B	1.61 µs / 880 B	1.12 µs / 1248 B	3.37 µs / 2208 B
deep	—	1.10 µs / 744 B	1.90 µs / 1712 B	767 ns / 1080 B	2.39 µs / 1392 B
wide-repeated	—	4.03 µs / 3184 B	4.35 µs / 2688 B	2.30 µs / 1096 B	7.39 µs / 4216 B
repeated-messages	2.31 µs / 2360 B	6.96 µs / 4560 B	13.53 µs / 9816 B	3.58 µs / 4024 B	17.23 µs / 10424 B
map-heavy	—	11.06 µs / 13984 B	25.12 µs / 25040 B	3.72 µs / 3600 B	12.44 µs / 10720 B

Decode (bytes → Clojure data)

shape	java	hinted	dynamic	jsonista	data.json
tiny	94 ns / 192 B	239 ns / 232 B	590 ns / 608 B	678 ns / 1176 B	591 ns / 1584 B
flat	346 ns / 432 B	1.18 µs / 600 B	1.95 µs / 1096 B	2.07 µs / 2248 B	3.27 µs / 5744 B
deep	—	1.54 µs / 1288 B	2.79 µs / 2480 B	1.42 µs / 2208 B	1.45 µs / 3840 B
wide-repeated	—	3.53 µs / 4344 B	6.90 µs / 4712 B	3.79 µs / 4144 B	3.06 µs / 11496 B
repeated-messages	2.07 µs / 3328 B	12.00 µs / 8448 B	18.25 µs / 15712 B	11.75 µs / 10920 B	12.01 µs / 24664 B
map-heavy	—	15.19 µs / 15432 B	28.91 µs / 29160 B	9.32 µs / 5816 B	14.59 µs / 21376 B

Read it honestly: the hinted arm sits ~2× off protoc's own generated code and ~3× ahead of DynamicMessage on small messages, beating JSON both ways there (typed-accessor invokers via LambdaMetafactory close most of the reflection gap; see docs/design.md); jackson wins on collection-heavy shapes, where per-entry message building dominates. Wire compactness and schema are protobuf's argument regardless. The shapes are archetypes precisely because no single number describes 'protobuf vs JSON'.

Building

Bazel (with rules_clj) is the build and test harness: bazel test //.... Plain clj works too: clojure -X:test. The Clojars artifact is produced by clojure -T:build jar from the same deps.edn and version.edn.

License

Apache-2.0