clj-protobuf

The protobuf runtime for code generated by protoc-gen-clojure: records in, protoc's bytes out. Editions supported through 2024.

;; deps.edn
com.github.bpalermo/clj-protobuf {:mvn/version "0.1.11"}

What it is

protoc-gen-clojure emits a defrecord per message plus X->proto / proto->X conversion fns; this library is everything those fns call — the descriptor machinery, the field codec, and byte-level encode/decode:

(require '[clj-protobuf.core :as pb]
         '[acme.greeter.greeter :as g])   ; generated

(-> (g/map->HelloRequest {:name "world" :repeat-count 2})
    g/HelloRequest->proto
    pb/encode)                            ; -> bytes, byte-identical to protoc's

(->> bytes
     (pb/decode g/HelloRequest-prototype)
     g/proto->HelloRequest)               ; -> record; absent fields are nil

Records and plain maps are interchangeable everywhere a message value goes.

Semantics worth knowing

• nil means absent, both directions. Setting nil sets nothing; reading an unset explicit-presence field returns nil. Fields with IMPLICIT presence (editions) and proto3 no-label scalars have no absence — they read back as their value, default included.
• The proto field name is the authority. Kebab-cased keys are derived from it (camelCaseField → :camel-case-field); STYLE_LEGACY files work because nothing ever converts backwards.
• Enums are keywords of the exact proto value name (:COLOR_RED), lossless in both directions; numbers, strings and EnumValueDescriptors are accepted on the way in.
• Editions are the descriptor's problem. The generated file embeds its FileDescriptorProto; protobuf-java resolves presence, DELIMITED encoding and utf8 validation when the descriptor builds. This library carries no edition-specific code, which is why new editions need a protobuf-java bump and nothing else.
• Unknown fields survive on parsed Messages but are necessarily dropped by a record round trip; pb/unknown-fields inspects them when it matters.

Performance

Generated code carries a Java-class hint per message. When the matching java_proto_library classes are on the classpath the prototypes silently switch from DynamicMessage to the generated classes, with byte-for-byte identical output (the byte-identity suite proves both arms against protoc's own Java backend).

Measured with bazel run //bench:run -- quick (criterium; JDK 21, Linux x86_64; mean latency / allocated bytes per op; full Clojure-data-to-bytes pipelines). java is protoc's generated builders driven directly; jsonista and data.json carry the same value as JSON:

Encode (Clojure data → bytes)

shape	java	hinted	dynamic	jsonista	data.json
tiny	56 ns / 56 B	141 ns / 136 B	489 ns / 448 B	257 ns / 480 B	935 ns / 624 B
flat	545 ns / 400 B	977 ns / 488 B	1.94 µs / 880 B	1.33 µs / 1248 B	3.27 µs / 2208 B
deep	—	744 ns / 552 B	1.90 µs / 1520 B	770 ns / 1080 B	2.37 µs / 1392 B
wide-repeated	—	4.12 µs / 3088 B	4.08 µs / 2592 B	2.64 µs / 1096 B	8.04 µs / 4216 B
repeated-messages	2.76 µs / 2312 B	5.30 µs / 2840 B	13.13 µs / 8504 B	4.16 µs / 4024 B	17.25 µs / 10424 B
map-heavy	—	9.86 µs / 7512 B	20.67 µs / 19000 B	4.00 µs / 3616 B	16.63 µs / 10720 B
enum-heavy	—	1.21 µs / 456 B	2.54 µs / 832 B	1.37 µs / 1168 B	4.24 µs / 2952 B

Decode (bytes → Clojure data)

shape	java	hinted	dynamic	jsonista	data.json
tiny	139 ns / 192 B	209 ns / 232 B	660 ns / 608 B	794 ns / 1136 B	643 ns / 1584 B
flat	388 ns / 432 B	923 ns / 600 B	2.10 µs / 1096 B	2.22 µs / 2168 B	3.90 µs / 5776 B
deep	—	762 ns / 1024 B	2.33 µs / 2216 B	1.47 µs / 2088 B	1.63 µs / 3840 B
wide-repeated	—	3.00 µs / 4296 B	6.59 µs / 4664 B	3.65 µs / 4104 B	3.54 µs / 11496 B
repeated-messages	2.47 µs / 3328 B	5.72 µs / 5360 B	15.49 µs / 12624 B	11.55 µs / 10080 B	11.10 µs / 25096 B
map-heavy	—	13.03 µs / 13048 B	30.23 µs / 29120 B	11.09 µs / 5736 B	18.89 µs / 22248 B
enum-heavy	—	2.58 µs / 1056 B	3.48 µs / 1440 B	2.79 µs / 2840 B	2.91 µs / 7200 B

Read it honestly: the hinted arm sits ~2× off protoc's own generated code and ~3× ahead of DynamicMessage on small messages, beating JSON both ways there (typed-accessor invokers via LambdaMetafactory close most of the reflection gap, and since 0.1.11 they cover repeated, map and open-enum fields too; see docs/design.md). Lists of messages now decode faster than JSON as well; jackson still wins encoding the collection-heavy shapes, where protobuf-java's own map and list building dominates. Wire compactness and schema are protobuf's argument regardless. The shapes are archetypes precisely because no single number describes 'protobuf vs JSON'.

Building

Bazel (with rules_clj) is the build and test harness: bazel test //.... Plain clj works too: clojure -X:test. The Clojars artifact is bazel build //src:clojars — jar and pom, from the same deps.edn and version.edn everything else here reads. bazel run //src:clojars.publish -- --dry-run prints every upload it would make.

License

Apache-2.0