The codec half of the generated-code contract: set-field! and get-field.
Generated ->proto fns call set-field! once per field against a fresh
builder; proto-> fns call get-field once per field feeding the record's
positional constructor. Both take an opts map that is almost always nil.
Semantics the whole design hangs on:
set-field! of nil sets nothing; get-field of an unset
explicit-presence field returns nil. Fields with IMPLICIT presence
(editions) and proto3 no-label scalars have no absence to report, so
get-field returns the value — default included — and never nil.->proto, so recursion happens here, through the handle's
child handles, which carry the concrete-class-correct nested prototypes.opts (all optional): :naming :kebab (default) | :proto — keys used on the generic map path :enums :keyword (default; exact proto value name, e.g. :GREETING_HELLO) | :number | :string — how get-field represents enums; set-field! accepts keyword, string, number or EnumValueDescriptor regardless :bytes :byte-array (default) | :byte-string
The codec half of the generated-code contract: `set-field!` and `get-field`.
Generated `->proto` fns call `set-field!` once per field against a fresh
builder; `proto->` fns call `get-field` once per field feeding the record's
positional constructor. Both take an opts map that is almost always nil.
Semantics the whole design hangs on:
- nil means absent, in both directions. A record has every key; protobuf has
presence. `set-field!` of nil sets nothing; `get-field` of an unset
explicit-presence field returns nil. Fields with IMPLICIT presence
(editions) and proto3 no-label scalars have no absence to report, so
`get-field` returns the value — default included — and never nil.
- Nested message values arrive as records or plain maps; generated code never
calls the nested `->proto`, so recursion happens here, through the handle's
child handles, which carry the concrete-class-correct nested prototypes.
- The proto field name is the authority. Kebab keys are derived; the reverse
mapping does not exist (STYLE_LEGACY).
opts (all optional):
:naming :kebab (default) | :proto — keys used on the generic map path
:enums :keyword (default; exact proto value name, e.g. :GREETING_HELLO)
| :number | :string — how get-field represents enums;
set-field! accepts keyword, string, number or EnumValueDescriptor
regardless
:bytes :byte-array (default) | :byte-stringSerialization, on top of what the generated code produces.
Generated X->proto / proto->X fns are Message-in/Message-out — they never
touch bytes. This namespace is the other half: Message to bytes and back.
Typical round trip:
(-> rec HelloRequest->proto pb/encode) ; -> bytes
(->> bytes (pb/decode HelloRequest-prototype) proto->HelloRequest)
decode takes the generated X-prototype var — the same value everything
else in the contract keys off — so there is exactly one handle per message
type in user code.
Serialization, on top of what the generated code produces.
Generated `X->proto` / `proto->X` fns are Message-in/Message-out — they never
touch bytes. This namespace is the other half: Message to bytes and back.
Typical round trip:
(-> rec HelloRequest->proto pb/encode) ; -> bytes
(->> bytes (pb/decode HelloRequest-prototype) proto->HelloRequest)
`decode` takes the generated `X-prototype` var — the same value everything
else in the contract keys off — so there is exactly one handle per message
type in user code.The codec compiler: a Descriptor walked once into everything the hot path needs, so that parsing and writing never touch the descriptor API, never resolve a FieldDescriptor, and never ask protobuf-java what an edition feature means — those questions are answered here, per type, the first time the type is used.
A message value is an Object[] of slots, one per field in descriptor order (FieldDescriptor.getIndex), plus an UnknownFieldSet. nil is absent. Fields without presence (proto3 scalars, editions IMPLICIT) are normalized so nil also means the default: readers and setters store nil for the default value, and reads substitute the default back. That keeps the writer's rule one line — write what is non-nil and non-empty — and makes re-encoding bytes that carried an explicit default drop it, as every protobuf implementation does.
Compiled per type:
fields: a CompiledField per slot, with the writer for that field.writers: the same fields in field-number order, the order every
protobuf serializer emits, followed by unknown fields.Nested types compile lazily through the parser-fn the message layer supplies, behind an IDeref, because descriptors are cyclic. One compiler owns one cache, keyed by Descriptor identity.
The codec compiler: a Descriptor walked once into everything the hot path needs, so that parsing and writing never touch the descriptor API, never resolve a FieldDescriptor, and never ask protobuf-java what an edition feature means — those questions are answered here, per type, the first time the type is used. A message value is an Object[] of slots, one per field in descriptor order (FieldDescriptor.getIndex), plus an UnknownFieldSet. nil is absent. Fields without presence (proto3 scalars, editions IMPLICIT) are normalized so nil also means the default: readers and setters store nil for the default value, and reads substitute the default back. That keeps the writer's rule one line — write what is non-nil and non-empty — and makes re-encoding bytes that carried an explicit default drop it, as every protobuf implementation does. Compiled per type: - `fields`: a CompiledField per slot, with the writer for that field. - `writers`: the same fields in field-number order, the order every protobuf serializer emits, followed by unknown fields. - a reader table keyed by TAG, not field number, dense (an array indexed by tag) when the tags allow and binary-searched otherwise. A repeated scalar registers both its packed and its expanded tag, because a parser accepts either encoding whatever the descriptor says it writes. - oneof membership, so reading a member off the wire clears its siblings, and the required slots, for initialization checks. Nested types compile lazily through the parser-fn the message layer supplies, behind an IDeref, because descriptors are cyclic. One compiler owns one cache, keyed by Descriptor identity.
Typed-accessor invokers, built once per field with LambdaMetafactory.
protobuf-java's reflection API pays a FieldAccessorTable lookup on every setField/getField. When the prototype is a generated class, the typed accessors (setFooBar, getFooBar, hasFooBar) are right there — and a metafactory-generated BiFunction/Function calling one runs at direct-interop speed (measured ~3.5 ns/op, vs ~µs through reflection paths), including the primitive boxing bridge the instantiated method type declares.
Everything here is best-effort by construction, in the same spirit as the Java-class hint: derive protoc's accessor name, let findVirtual verify it exists with the expected signature, and return nil on ANY failure — including LambdaMetafactory itself being unavailable, which is what happens under native-image, where the codec silently keeps its reflection path. A wrong derivation is never wrong bytes, only a missed optimisation.
Typed-accessor invokers, built once per field with LambdaMetafactory. protobuf-java's reflection API pays a FieldAccessorTable lookup on every setField/getField. When the prototype is a generated class, the typed accessors (setFooBar, getFooBar, hasFooBar) are right there — and a metafactory-generated BiFunction/Function calling one runs at direct-interop speed (measured ~3.5 ns/op, vs ~µs through reflection paths), including the primitive boxing bridge the instantiated method type declares. Everything here is best-effort by construction, in the same spirit as the Java-class hint: derive protoc's accessor name, let findVirtual verify it exists with the expected signature, and return nil on ANY failure — including LambdaMetafactory itself being unavailable, which is what happens under native-image, where the codec silently keeps its reflection path. A wrong derivation is never wrong bytes, only a missed optimisation.
The compiled message: three deftypes implementing protobuf-java's
Message, Message.Builder and Parser interfaces over the compiler's slot
layout, so that everything downstream — the generated code's
.newBuilderForType / .build, core/encode and core/decode, grpc
marshallers, TextFormat and JsonFormat — keeps working, with none of
DynamicMessage's FieldSet behind it.
A CompiledMessage is a compiled type, an Object[] of slots and an UnknownFieldSet (nil when empty). It is immutable; its serialized size is memoized. A CompiledBuilder is the same, mutable. Building hands the slot array to the message rather than copying it, so a builder that has built, or was made by toBuilder or clone, owns nothing: it copies the array and its collections before its next write. See SlotOwner. The parser owns fresh slots, so parsing mutates freely.
The reflective API returns what protobuf-java's does — EnumValueDescriptor for enums, a list of entry messages for maps, the nested default instance for an unset message field — and equals/hashCode follow AbstractMessage's algorithm exactly, so a compiled message equals and hashes like a DynamicMessage of the same descriptor and value. Descriptors from another pool (a generated class) are never equal, by protobuf-java's own rule.
The compiled message: three deftypes implementing protobuf-java's Message, Message.Builder and Parser interfaces over the compiler's slot layout, so that everything downstream — the generated code's `.newBuilderForType` / `.build`, `core/encode` and `core/decode`, grpc marshallers, TextFormat and JsonFormat — keeps working, with none of DynamicMessage's FieldSet behind it. A CompiledMessage is a compiled type, an Object[] of slots and an UnknownFieldSet (nil when empty). It is immutable; its serialized size is memoized. A CompiledBuilder is the same, mutable. Building hands the slot array to the message rather than copying it, so a builder that has built, or was made by toBuilder or clone, owns nothing: it copies the array and its collections before its next write. See SlotOwner. The parser owns fresh slots, so parsing mutates freely. The reflective API returns what protobuf-java's does — EnumValueDescriptor for enums, a list of entry messages for maps, the nested default instance for an unset message field — and equals/hashCode follow AbstractMessage's algorithm exactly, so a compiled message equals and hashes like a DynamicMessage of the same descriptor and value. Descriptors from another pool (a generated class) are never equal, by protobuf-java's own rule.
The one naming rule, shared with the emitter.
protoc-gen-clojure kebab-cases proto field names into record fields and map
keys with exactly this algorithm (its field-key-symbol). The runtime's
generic nested-map path must produce the same keys byte for byte, or records
built by generated code and maps built by the runtime stop being
interchangeable. Any change here is a wire-compatibility break with every
generated file in existence — don't.
Kebab-casing is lossy (STYLE_LEGACY files can mix conventions), which is why
the emitted rt/field lookups carry the exact proto name and this fn is used
only for the Clojure-side keys.
The one naming rule, shared with the emitter. protoc-gen-clojure kebab-cases proto field names into record fields and map keys with exactly this algorithm (its `field-key-symbol`). The runtime's generic nested-map path must produce the same keys byte for byte, or records built by generated code and maps built by the runtime stop being interchangeable. Any change here is a wire-compatibility break with every generated file in existence — don't. Kebab-casing is lossy (STYLE_LEGACY files can mix conventions), which is why the emitted `rt/field` lookups carry the exact proto name and this fn is used only for the Clojure-side keys.
Wire primitives for the compiled codec: one writer and one reader per field, built once from the field's type and number, over protobuf-java's own CodedOutputStream and CodedInputStream. Varints, zigzag, fixed widths, UTF-8 and the length arithmetic stay protobuf-java's code; what this namespace adds is the choice of which call to make, made once per field instead of once per value.
A writer writes one field — tag included, every element for repeateds, one length-delimited run when packed — and reports its serialized size. A reader is handed the input positioned just after this field's tag and the slot's current value, and returns the slot's new value: the scalar, the list with one more element, the map with one more entry, or a merged message when a singular message field repeats on the wire. Readers are keyed by tag rather than by field number upstream, because a repeated scalar may arrive packed or expanded whatever its descriptor says it writes, and the two are different tags.
Slot representations, shared with the compiler and the codec: int32 kinds Integer int64 kinds Long float Float double Double bool Boolean string String bytes ByteString enum Integer (the number) message Message repeated java.util.ArrayList map java.util.LinkedHashMap (insertion order is wire order)
Wire primitives for the compiled codec: one writer and one reader per field, built once from the field's type and number, over protobuf-java's own CodedOutputStream and CodedInputStream. Varints, zigzag, fixed widths, UTF-8 and the length arithmetic stay protobuf-java's code; what this namespace adds is the choice of which call to make, made once per field instead of once per value. A writer writes one field — tag included, every element for repeateds, one length-delimited run when packed — and reports its serialized size. A reader is handed the input positioned just after this field's tag and the slot's current value, and returns the slot's new value: the scalar, the list with one more element, the map with one more entry, or a merged message when a singular message field repeats on the wire. Readers are keyed by tag rather than by field number upstream, because a repeated scalar may arrive packed or expanded whatever its descriptor says it writes, and the two are different tags. Slot representations, shared with the compiler and the codec: int32 kinds Integer int64 kinds Long float Float double Double bool Boolean string String bytes ByteString enum Integer (the number) message Message repeated java.util.ArrayList map java.util.LinkedHashMap (insertion order is wire order)
The runtime half of protoc-gen-clojure's generated-code contract.
Generated files call exactly four things here: file-descriptor to rebuild
their embedded FileDescriptorProto, known-file for well-known-type deps,
message for a prototype per message, and field for a handle per field.
Everything protobuf decides per edition — presence, delimited encoding, utf8
validation, packedness — is resolved by protobuf-java when the descriptor is
built, which is why generated code never mentions editions at all.
field returns a precomputed FieldHandle rather than a bare FieldDescriptor:
the codec's hot path dispatches on a keyword and never touches the descriptor
API per call. Handles are built against a specific prototype, so a
message-typed field's nested prototype has the right concrete class in both
the DynamicMessage and generated-class arms.
The runtime half of protoc-gen-clojure's generated-code contract. Generated files call exactly four things here: `file-descriptor` to rebuild their embedded FileDescriptorProto, `known-file` for well-known-type deps, `message` for a prototype per message, and `field` for a handle per field. Everything protobuf decides per edition — presence, delimited encoding, utf8 validation, packedness — is resolved by protobuf-java when the descriptor is built, which is why generated code never mentions editions at all. `field` returns a precomputed FieldHandle rather than a bare FieldDescriptor: the codec's hot path dispatches on a keyword and never touches the descriptor API per call. Handles are built against a specific prototype, so a message-typed field's nested prototype has the right concrete class in both the DynamicMessage and generated-class arms.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |