The codec compiler: a Descriptor walked once into everything the hot path needs, so that parsing and writing never touch the descriptor API, never resolve a FieldDescriptor, and never ask protobuf-java what an edition feature means — those questions are answered here, per type, the first time the type is used.
A message value is an Object[] of slots, one per field in descriptor order (FieldDescriptor.getIndex), plus an UnknownFieldSet. nil is absent. Fields without presence (proto3 scalars, editions IMPLICIT) are normalized so nil also means the default: readers and setters store nil for the default value, and reads substitute the default back. That keeps the writer's rule one line — write what is non-nil and non-empty — and makes re-encoding bytes that carried an explicit default drop it, as every protobuf implementation does.
Compiled per type:
fields: a CompiledField per slot, with the writer for that field.writers: the same fields in field-number order, the order every
protobuf serializer emits, followed by unknown fields.Nested types compile lazily through the parser-fn the message layer supplies, behind an IDeref, because descriptors are cyclic. One compiler owns one cache, keyed by Descriptor identity.
The codec compiler: a Descriptor walked once into everything the hot path needs, so that parsing and writing never touch the descriptor API, never resolve a FieldDescriptor, and never ask protobuf-java what an edition feature means — those questions are answered here, per type, the first time the type is used. A message value is an Object[] of slots, one per field in descriptor order (FieldDescriptor.getIndex), plus an UnknownFieldSet. nil is absent. Fields without presence (proto3 scalars, editions IMPLICIT) are normalized so nil also means the default: readers and setters store nil for the default value, and reads substitute the default back. That keeps the writer's rule one line — write what is non-nil and non-empty — and makes re-encoding bytes that carried an explicit default drop it, as every protobuf implementation does. Compiled per type: - `fields`: a CompiledField per slot, with the writer for that field. - `writers`: the same fields in field-number order, the order every protobuf serializer emits, followed by unknown fields. - a reader table keyed by TAG, not field number, dense (an array indexed by tag) when the tags allow and binary-searched otherwise. A repeated scalar registers both its packed and its expanded tag, because a parser accepts either encoding whatever the descriptor says it writes. - oneof membership, so reading a member off the wire clears its siblings, and the required slots, for initialization checks. Nested types compile lazily through the parser-fn the message layer supplies, behind an IDeref, because descriptors are cyclic. One compiler owns one cache, keyed by Descriptor identity.
Typed-accessor invokers, built once per field with LambdaMetafactory.
protobuf-java's reflection API pays a FieldAccessorTable lookup on every setField/getField. When the prototype is a generated class, the typed accessors (setFooBar, getFooBar, hasFooBar) are right there — and a metafactory-generated BiFunction/Function calling one runs at direct-interop speed (measured ~3.5 ns/op, vs ~µs through reflection paths), including the primitive boxing bridge the instantiated method type declares.
Everything here is best-effort by construction, in the same spirit as the Java-class hint: derive protoc's accessor name, let findVirtual verify it exists with the expected signature, and return nil on ANY failure — including LambdaMetafactory itself being unavailable, which is what happens under native-image, where the codec silently keeps its reflection path. A wrong derivation is never wrong bytes, only a missed optimisation.
Typed-accessor invokers, built once per field with LambdaMetafactory. protobuf-java's reflection API pays a FieldAccessorTable lookup on every setField/getField. When the prototype is a generated class, the typed accessors (setFooBar, getFooBar, hasFooBar) are right there — and a metafactory-generated BiFunction/Function calling one runs at direct-interop speed (measured ~3.5 ns/op, vs ~µs through reflection paths), including the primitive boxing bridge the instantiated method type declares. Everything here is best-effort by construction, in the same spirit as the Java-class hint: derive protoc's accessor name, let findVirtual verify it exists with the expected signature, and return nil on ANY failure — including LambdaMetafactory itself being unavailable, which is what happens under native-image, where the codec silently keeps its reflection path. A wrong derivation is never wrong bytes, only a missed optimisation.
The compiled message: three deftypes implementing protobuf-java's
Message, Message.Builder and Parser interfaces over the compiler's slot
layout, so that everything downstream — the generated code's
.newBuilderForType / .build, core/encode and core/decode, grpc
marshallers, TextFormat and JsonFormat — keeps working, with none of
DynamicMessage's FieldSet behind it.
A CompiledMessage is a compiled type, an Object[] of slots and an UnknownFieldSet (nil when empty). It is immutable; its serialized size is memoized. A CompiledBuilder is the same, mutable. Collections in slots are never mutated in place once a message may share them: a builder that has built, or was made by toBuilder, copies a collection before touching it. The parser owns fresh slots, so parsing mutates freely.
The reflective API returns what protobuf-java's does — EnumValueDescriptor for enums, a list of entry messages for maps, the nested default instance for an unset message field — and equals/hashCode follow AbstractMessage's algorithm exactly, so a compiled message equals and hashes like a DynamicMessage of the same descriptor and value. Descriptors from another pool (a generated class) are never equal, by protobuf-java's own rule.
The compiled message: three deftypes implementing protobuf-java's Message, Message.Builder and Parser interfaces over the compiler's slot layout, so that everything downstream — the generated code's `.newBuilderForType` / `.build`, `core/encode` and `core/decode`, grpc marshallers, TextFormat and JsonFormat — keeps working, with none of DynamicMessage's FieldSet behind it. A CompiledMessage is a compiled type, an Object[] of slots and an UnknownFieldSet (nil when empty). It is immutable; its serialized size is memoized. A CompiledBuilder is the same, mutable. Collections in slots are never mutated in place once a message may share them: a builder that has built, or was made by toBuilder, copies a collection before touching it. The parser owns fresh slots, so parsing mutates freely. The reflective API returns what protobuf-java's does — EnumValueDescriptor for enums, a list of entry messages for maps, the nested default instance for an unset message field — and equals/hashCode follow AbstractMessage's algorithm exactly, so a compiled message equals and hashes like a DynamicMessage of the same descriptor and value. Descriptors from another pool (a generated class) are never equal, by protobuf-java's own rule.
The one naming rule, shared with the emitter.
protoc-gen-clojure kebab-cases proto field names into record fields and map
keys with exactly this algorithm (its field-key-symbol). The runtime's
generic nested-map path must produce the same keys byte for byte, or records
built by generated code and maps built by the runtime stop being
interchangeable. Any change here is a wire-compatibility break with every
generated file in existence — don't.
Kebab-casing is lossy (STYLE_LEGACY files can mix conventions), which is why
the emitted rt/field lookups carry the exact proto name and this fn is used
only for the Clojure-side keys.
The one naming rule, shared with the emitter. protoc-gen-clojure kebab-cases proto field names into record fields and map keys with exactly this algorithm (its `field-key-symbol`). The runtime's generic nested-map path must produce the same keys byte for byte, or records built by generated code and maps built by the runtime stop being interchangeable. Any change here is a wire-compatibility break with every generated file in existence — don't. Kebab-casing is lossy (STYLE_LEGACY files can mix conventions), which is why the emitted `rt/field` lookups carry the exact proto name and this fn is used only for the Clojure-side keys.
Wire primitives for the compiled codec: one writer and one reader per field, built once from the field's type and number, over protobuf-java's own CodedOutputStream and CodedInputStream. Varints, zigzag, fixed widths, UTF-8 and the length arithmetic stay protobuf-java's code; what this namespace adds is the choice of which call to make, made once per field instead of once per value.
A writer writes one field — tag included, every element for repeateds, one length-delimited run when packed — and reports its serialized size. A reader is handed the input positioned just after this field's tag and the slot's current value, and returns the slot's new value: the scalar, the list with one more element, the map with one more entry, or a merged message when a singular message field repeats on the wire. Readers are keyed by tag rather than by field number upstream, because a repeated scalar may arrive packed or expanded whatever its descriptor says it writes, and the two are different tags.
Slot representations, shared with the compiler and the codec: int32 kinds Integer int64 kinds Long float Float double Double bool Boolean string String bytes ByteString enum Integer (the number) message Message repeated java.util.ArrayList map java.util.LinkedHashMap (insertion order is wire order)
Wire primitives for the compiled codec: one writer and one reader per field, built once from the field's type and number, over protobuf-java's own CodedOutputStream and CodedInputStream. Varints, zigzag, fixed widths, UTF-8 and the length arithmetic stay protobuf-java's code; what this namespace adds is the choice of which call to make, made once per field instead of once per value. A writer writes one field — tag included, every element for repeateds, one length-delimited run when packed — and reports its serialized size. A reader is handed the input positioned just after this field's tag and the slot's current value, and returns the slot's new value: the scalar, the list with one more element, the map with one more entry, or a merged message when a singular message field repeats on the wire. Readers are keyed by tag rather than by field number upstream, because a repeated scalar may arrive packed or expanded whatever its descriptor says it writes, and the two are different tags. Slot representations, shared with the compiler and the codec: int32 kinds Integer int64 kinds Long float Float double Double bool Boolean string String bytes ByteString enum Integer (the number) message Message repeated java.util.ArrayList map java.util.LinkedHashMap (insertion order is wire order)
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |