Liking cljdoc? Tell your friends :D

clj-protobuf.impl.compile

The codec compiler: a Descriptor walked once into everything the hot path needs, so that parsing and writing never touch the descriptor API, never resolve a FieldDescriptor, and never ask protobuf-java what an edition feature means — those questions are answered here, per type, the first time the type is used.

A message value is an Object[] of slots, one per field in descriptor order (FieldDescriptor.getIndex), plus an UnknownFieldSet. nil is absent. Fields without presence (proto3 scalars, editions IMPLICIT) are normalized so nil also means the default: readers and setters store nil for the default value, and reads substitute the default back. That keeps the writer's rule one line — write what is non-nil and non-empty — and makes re-encoding bytes that carried an explicit default drop it, as every protobuf implementation does.

Compiled per type:

  • fields: a CompiledField per slot, with the writer for that field.
  • writers: the same fields in field-number order, the order every protobuf serializer emits, followed by unknown fields.
  • a reader table keyed by TAG, not field number, dense (an array indexed by tag) when the tags allow and binary-searched otherwise. A repeated scalar registers both its packed and its expanded tag, because a parser accepts either encoding whatever the descriptor says it writes.
  • oneof membership, so reading a member off the wire clears its siblings, and the required slots, for initialization checks.

Nested types compile lazily through the parser-fn the message layer supplies, behind an IDeref, because descriptors are cyclic. One compiler owns one cache, keyed by Descriptor identity.

The codec compiler: a Descriptor walked once into everything the hot
path needs, so that parsing and writing never touch the descriptor API,
never resolve a FieldDescriptor, and never ask protobuf-java what an
edition feature means — those questions are answered here, per type, the
first time the type is used.

A message value is an Object[] of slots, one per field in descriptor
order (FieldDescriptor.getIndex), plus an UnknownFieldSet. nil is absent.
Fields without presence (proto3 scalars, editions IMPLICIT) are normalized
so nil also means the default: readers and setters store nil for the
default value, and reads substitute the default back. That keeps the
writer's rule one line — write what is non-nil and non-empty — and makes
re-encoding bytes that carried an explicit default drop it, as every
protobuf implementation does.

Compiled per type:
- `fields`: a CompiledField per slot, with the writer for that field.
- `writers`: the same fields in field-number order, the order every
  protobuf serializer emits, followed by unknown fields.
- a reader table keyed by TAG, not field number, dense (an array indexed
  by tag) when the tags allow and binary-searched otherwise. A repeated
  scalar registers both its packed and its expanded tag, because a parser
  accepts either encoding whatever the descriptor says it writes.
- oneof membership, so reading a member off the wire clears its siblings,
  and the required slots, for initialization checks.

Nested types compile lazily through the parser-fn the message layer
supplies, behind an IDeref, because descriptors are cyclic. One compiler
owns one cache, keyed by Descriptor identity.
raw docstring

clj-protobuf.impl.invoke

Typed-accessor invokers, built once per field with LambdaMetafactory.

protobuf-java's reflection API pays a FieldAccessorTable lookup on every setField/getField. When the prototype is a generated class, the typed accessors (setFooBar, getFooBar, hasFooBar) are right there — and a metafactory-generated BiFunction/Function calling one runs at direct-interop speed (measured ~3.5 ns/op, vs ~µs through reflection paths), including the primitive boxing bridge the instantiated method type declares.

Everything here is best-effort by construction, in the same spirit as the Java-class hint: derive protoc's accessor name, let findVirtual verify it exists with the expected signature, and return nil on ANY failure — including LambdaMetafactory itself being unavailable, which is what happens under native-image, where the codec silently keeps its reflection path. A wrong derivation is never wrong bytes, only a missed optimisation.

Typed-accessor invokers, built once per field with LambdaMetafactory.

protobuf-java's reflection API pays a FieldAccessorTable lookup on every
setField/getField. When the prototype is a generated class, the typed
accessors (setFooBar, getFooBar, hasFooBar) are right there — and a
metafactory-generated BiFunction/Function calling one runs at direct-interop
speed (measured ~3.5 ns/op, vs ~µs through reflection paths), including the
primitive boxing bridge the instantiated method type declares.

Everything here is best-effort by construction, in the same spirit as the
Java-class hint: derive protoc's accessor name, let findVirtual verify it
exists with the expected signature, and return nil on ANY failure —
including LambdaMetafactory itself being unavailable, which is what happens
under native-image, where the codec silently keeps its reflection path.
A wrong derivation is never wrong bytes, only a missed optimisation.
raw docstring

clj-protobuf.impl.message

The compiled message: three deftypes implementing protobuf-java's Message, Message.Builder and Parser interfaces over the compiler's slot layout, so that everything downstream — the generated code's .newBuilderForType / .build, core/encode and core/decode, grpc marshallers, TextFormat and JsonFormat — keeps working, with none of DynamicMessage's FieldSet behind it.

A CompiledMessage is a compiled type, an Object[] of slots and an UnknownFieldSet (nil when empty). It is immutable; its serialized size is memoized. A CompiledBuilder is the same, mutable. Collections in slots are never mutated in place once a message may share them: a builder that has built, or was made by toBuilder, copies a collection before touching it. The parser owns fresh slots, so parsing mutates freely.

The reflective API returns what protobuf-java's does — EnumValueDescriptor for enums, a list of entry messages for maps, the nested default instance for an unset message field — and equals/hashCode follow AbstractMessage's algorithm exactly, so a compiled message equals and hashes like a DynamicMessage of the same descriptor and value. Descriptors from another pool (a generated class) are never equal, by protobuf-java's own rule.

The compiled message: three deftypes implementing protobuf-java's
Message, Message.Builder and Parser interfaces over the compiler's slot
layout, so that everything downstream — the generated code's
`.newBuilderForType` / `.build`, `core/encode` and `core/decode`, grpc
marshallers, TextFormat and JsonFormat — keeps working, with none of
DynamicMessage's FieldSet behind it.

A CompiledMessage is a compiled type, an Object[] of slots and an
UnknownFieldSet (nil when empty). It is immutable; its serialized size is
memoized. A CompiledBuilder is the same, mutable. Collections in slots are
never mutated in place once a message may share them: a builder that has
built, or was made by toBuilder, copies a collection before touching it.
The parser owns fresh slots, so parsing mutates freely.

The reflective API returns what protobuf-java's does — EnumValueDescriptor
for enums, a list of entry messages for maps, the nested default instance
for an unset message field — and equals/hashCode follow AbstractMessage's
algorithm exactly, so a compiled message equals and hashes like a
DynamicMessage of the same descriptor and value. Descriptors from another
pool (a generated class) are never equal, by protobuf-java's own rule.
raw docstring

clj-protobuf.impl.naming

The one naming rule, shared with the emitter.

protoc-gen-clojure kebab-cases proto field names into record fields and map keys with exactly this algorithm (its field-key-symbol). The runtime's generic nested-map path must produce the same keys byte for byte, or records built by generated code and maps built by the runtime stop being interchangeable. Any change here is a wire-compatibility break with every generated file in existence — don't.

Kebab-casing is lossy (STYLE_LEGACY files can mix conventions), which is why the emitted rt/field lookups carry the exact proto name and this fn is used only for the Clojure-side keys.

The one naming rule, shared with the emitter.

protoc-gen-clojure kebab-cases proto field names into record fields and map
keys with exactly this algorithm (its `field-key-symbol`). The runtime's
generic nested-map path must produce the same keys byte for byte, or records
built by generated code and maps built by the runtime stop being
interchangeable. Any change here is a wire-compatibility break with every
generated file in existence — don't.

Kebab-casing is lossy (STYLE_LEGACY files can mix conventions), which is why
the emitted `rt/field` lookups carry the exact proto name and this fn is used
only for the Clojure-side keys.
raw docstring

clj-protobuf.impl.wire

Wire primitives for the compiled codec: one writer and one reader per field, built once from the field's type and number, over protobuf-java's own CodedOutputStream and CodedInputStream. Varints, zigzag, fixed widths, UTF-8 and the length arithmetic stay protobuf-java's code; what this namespace adds is the choice of which call to make, made once per field instead of once per value.

A writer writes one field — tag included, every element for repeateds, one length-delimited run when packed — and reports its serialized size. A reader is handed the input positioned just after this field's tag and the slot's current value, and returns the slot's new value: the scalar, the list with one more element, the map with one more entry, or a merged message when a singular message field repeats on the wire. Readers are keyed by tag rather than by field number upstream, because a repeated scalar may arrive packed or expanded whatever its descriptor says it writes, and the two are different tags.

Slot representations, shared with the compiler and the codec: int32 kinds Integer int64 kinds Long float Float double Double bool Boolean string String bytes ByteString enum Integer (the number) message Message repeated java.util.ArrayList map java.util.LinkedHashMap (insertion order is wire order)

Wire primitives for the compiled codec: one writer and one reader per
field, built once from the field's type and number, over protobuf-java's
own CodedOutputStream and CodedInputStream. Varints, zigzag, fixed widths,
UTF-8 and the length arithmetic stay protobuf-java's code; what this
namespace adds is the choice of which call to make, made once per field
instead of once per value.

A writer writes one field — tag included, every element for repeateds, one
length-delimited run when packed — and reports its serialized size. A
reader is handed the input positioned just after this field's tag and the
slot's current value, and returns the slot's new value: the scalar, the
list with one more element, the map with one more entry, or a merged
message when a singular message field repeats on the wire. Readers are
keyed by tag rather than by field number upstream, because a repeated
scalar may arrive packed or expanded whatever its descriptor says it
writes, and the two are different tags.

Slot representations, shared with the compiler and the codec:
  int32 kinds  Integer      int64 kinds  Long
  float        Float        double       Double
  bool         Boolean      string       String
  bytes        ByteString   enum         Integer (the number)
  message      Message      repeated     java.util.ArrayList
  map          java.util.LinkedHashMap (insertion order is wire order)
raw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close