Release v0.1.10 — every binding ships this one version; VERSION is the source of truth, scripts/sync-versions.sh propagates it.

pbwire — a dynamic Protobuf runtime for C, no code generation

pbwire is a dependency-free Protobuf runtime written in C. It loads a schema at runtime — a hand-built descriptor tree or a standard FileDescriptorSet — then builds, reads, encodes and decodes messages dynamically through reflection. No protoc step, no generated message code, no libprotobuf in the process. It speaks proto2, proto3 and edition-2023 descriptor sets, the JSON mapping and the text format, and drives fifteen host-language bindings from that one core.

The point is not to re-implement Protobuf encoding. It is to let every language with a C ABI share one dynamic Protobuf schema and one message-processing core instead of re-implementing Protobuf per host language.

uint8_t *buf;
size_t len;
pbw_encode(msg, &buf, &len);   /* 0a 03 61 62 63 10 ff ff ff ff ff ff ff ff ff 01 */
pbw_free(buf);

Documentation

Document	Contents
C API guide	the tutorial, hand-built schema, FileDescriptorSet load, pbw_ctx loop, ownership rules, streaming, unknown fields, JSON/text, ABI
Design and architecture	how it works, dynamic message semantics, the two allocation strategies, scope, limits, use cases, repository layout
Performance benchmarks	the performance document: the four standard tiers, every binding against its official runtime, and the cross-binding comparison
Testing and conformance	the self-check, pinned reference bytes, difftest, the official conformance suite
FAQ	syntax support, thread safety, unknown fields, untrusted input, .proto at runtime
Protobuf pain points	which official-library complaints pbwire solves, eases or leaves alone — the searchable index
Bindings	every host binding, where its schema comes from, and the per-language pain point each one removes
include/pbwire.h	the whole public API with per-function contracts
AGENTS.md · CONTRIBUTING.md · llms.txt	working on this repo, commit rules, a one-page factual summary

Why use pbwire

• Schema at runtime. Load a FileDescriptorSet produced by protoc, buf or pbwire-protoc, and handle message types you did not compile in.
• No runtime dependencies. C99, one public header, ~13k lines of C (about 7.5k of it the binary codec, the rest the JSON mapping and text format). make and make test need nothing but a compiler — no libprotobuf, no protoc, no JSON/C++ runtime underneath.
• Dynamic messages and reflection. Create messages from a descriptor, get/set fields by number or name, walk nested structures, oneofs and maps.
• Byte-identical to the reference implementation. Encodings match protoc 36.2 for every scalar type, packed/unpacked repetition and nested messages; pinned bytes in the test suite keep parity from regressing silently.
• Built for embedding. Stable C ABI, no globals, caller-confined schema/message objects, arena-backed pbw_ctx for per-thread hot loops, wire input validated at the boundary.
• One core, twenty-three host languages. C, C++, Python, Ruby, Java, Kotlin, Node, Objective-C, PHP, Dart, C#, Rust, Go, Erlang, Lua, Swift, Scala, Elixir, Zig, F#, Clojure, Julia and R all speak the same wire format, each with a measured answer to its runtime's own pain point — see what each binding solves.

At a glance

Language	C99 (C++ consumers include the same header)
Wire format	Google Protobuf binary, proto2 + proto3 (edition-2023 descriptor sets load too)
Schema input	hand-built descriptors, or a standard FileDescriptorSet
Message handling	dynamic value tree with reflection; no generated code
Runtime dependencies	none (optional importer pbwire-protoc needs libprotobuf, C++17)
Not implemented	RPC, MessageSet decoding into extensions
Memory model	one allocation per decoded tree, or one reusable arena pbw_ctx per thread
Limits	nesting depth 256, field numbers up to 2^29-1
Platforms	Linux, macOS, anywhere with a C99 compiler
License	Apache-2.0

Requirements and build

A C99 compiler for the core. CMake plus a C++17 compiler and libprotobuf only for pbwire-protoc.

make          # build build/libpbwire.a
make test     # self-checks: core, threads, descriptor set, C structs, examples round trip
make shared   # libpbwire.so.1 / libpbwire.1.dylib, for FFI hosts that dlopen it
make format   # clang-format (>= 19); make format-check
make clean

CMake is the packaging path and the only build that knows about pbwire-protoc:

cmake -B build-cmake                        # core only, no protobuf needed
cmake -B build-cmake -DPBW_BUILD_PROTOC=ON   # core + .proto importer (needs libprotobuf)
cmake --build build-cmake
(cd build-cmake && ctest)                    # same checks as `make test`, plus test_importer

# installed targets: pbwire::pbwire and pbwire::protoc
cmake --install build-cmake --prefix /usr/local

# then, in a consumer:
# find_package(pbwire REQUIRED)
# target_link_libraries(app PRIVATE pbwire::pbwire)

Everything else is optional and outside the commit gate: make difftest, make conformance and the per-binding *-test / *-bench targets. See docs/testing.md for what they verify and when they are worth running; binding targets are listed in each binding's README and in BENCHMARKS.md.

Quick start

#include "pbwire.h"

pbw_schema *schema = pbw_schema_new();
pbw_descriptor *person = pbw_schema_add_message(schema, "Person");
pbw_message_add_field(person, "name", 1, PBW_TYPE_STRING, PBW_LABEL_OPTIONAL);
pbw_message_add_field(person, "id", 2, PBW_TYPE_INT32, PBW_LABEL_OPTIONAL);
pbw_schema_link(schema);

pbw_message *msg = pbw_message_new(person);
pbw_message_set(msg, 1, pbw_value_new_string("abc"));
pbw_message_set(msg, 2, pbw_value_new_int(-1));

uint8_t *buf;
size_t len;
pbw_encode(msg, &buf, &len);   /* 0a 03 61 62 63 10 ff ff ff ff ff ff ff ff ff 01 */

pbw_message *decoded;
pbw_decode(person, buf, len, &decoded);
const char *name = pbw_value_get_string(pbw_message_get(decoded, 1));

pbw_free(buf);
pbw_message_free(decoded);
pbw_message_free(msg);
pbw_schema_free(schema);

Real programs rarely build a schema by hand — they load a FileDescriptorSet at runtime and process types they were never compiled against. The walk-through for that, the arena-backed hot loop, streaming framing and the JSON/text entry points is docs/guide.md; a runnable tutorial schema (proto2 address book, loaded from addressbook.desc at runtime) is in examples/ and runs with make examples.

Performance

make bench verifies byte-for-byte equality with libprotobuf before timing anything, so a speed number can never come from a wrong encoder. On an Apple M4 Pro with -O2 and protoc 36.2, the in-process dynamic core with a pbw_ctx runs 1.1-1.7x the throughput of the generated C++ codec on every tier (132 B to 86 KB, encode and decode), and 10-16x the speed of C++ DynamicMessage on encode and 3-4x on decode. Generated code only wins where pbwire's plain API pays one malloc/free pair per call (about parity at the small tiers, either way within run noise) or where a dynamic codec meets generated straight-line writes on a tiny message (Java/Kotlin, 136 B, 0.69x/0.82x); the arena API wins every tier.

The per-language tables — the same four payload tiers for every binding against its official runtime (Python, Java, Kotlin, Node, Ruby, Objective-C, PHP, Dart, Rust, C#, Erlang, Swift, Scala, Elixir, F#, Clojure, Julia), plus the fastest third-party runtime wherever one meaningfully exists (Node, Rust, C#, PHP), the C struct codec, 1-vs-8-thread scaling, and the reason behind each gap — live in BENCHMARKS.md. The one-row summary is here; the detail is there.

Language	How it talks to the core	Best encode / decode vs official	Baseline
C	static link, struct codec	1.31x (xlarge) / 2.33x (xlarge)	protoc C++
C++	same C runtime, same header	1.31x (xlarge) / 2.33x (xlarge)¹	protoc C++
Python	CPython extension	6.7x (xlarge) / 1.72x (xlarge)	google.protobuf (upb)
Ruby	C extension, linked	1.42x (small) / 1.86x (large)	google-protobuf gem
Java	pure-Java codec, no native code	1.24x (medium) / 2.40x (xlarge)	protobuf-java
Kotlin	Java runtime + DSL	1.27x (medium) / 2.01x (xlarge)	protobuf-kotlin
Node	N-API addon	25.7x (xlarge) / 18.6x (xlarge)	jspb + protobufjs
Objective-C	ARC wrapper	6.5x (small) / 3.2x (large)	protobuf-objc
PHP	C extension	1.10x (xlarge) / 1.25x (xlarge) vs C ext (344x / 134x vs pure PHP)	protobuf-php
Dart	dart:ffi	12.2x (xlarge) / 3.3x (xlarge)	protobuf-dart
Rust	static link	2.05x (large) / 1.70x (large)	rust-protobuf 3.x + prost
C#	P/Invoke	3.32x (medium) / 2.40x (small)	Google.Protobuf + protobuf-net
Go	cgo dynamic (hot-swappable registry)	2.63x (xlarge) / 2.06x (large)	google.golang.org/protobuf
Erlang	NIF	4.03x (xlarge) / 2.21x (medium)	gpb
Lua	C module, tables + builder	7.94x (xlarge, builder) / 1.16x (large)	lua-protobuf
Swift	SwiftPM, static link	5.63x (xlarge) / 4.44x (medium)	swift-protobuf
Scala	Java runtime + DSL	1.43x (small) / 1.65x (xlarge)	ScalaPB
Elixir	NIF (shared core)	3.16x (small) / 1.68x (large)	elixir-protobuf
Zig	static link	no baseline wired yet	zig-protobuf
F#	C# runtime + facade	3.30x (medium) / 2.46x (medium)	Google.Protobuf
Clojure	Java runtime + interop	1.30x (xlarge) / 2.07x (xlarge)	protobuf-java
Julia	dlopen + ccall	137x (xlarge) / 1.10x (large)	ProtoBuf.jl
R	C extension, named lists	no baseline installed	RProtoBuf

¹ C++ has no separate bench; same engine as the C struct codec. Best columns take the best ratio across the four tiers for encode/decode separately, tier in parentheses (Apple M4 Pro, protoc 36.2); tiers and absolute MB/s are in BENCHMARKS.md.

Comparison with generated code, DynamicMessage and upb

	Schema at runtime	Generated code	Runtime dependency	Dynamic messages + reflection
pbwire	yes	none	none (C99)	yes
protoc generated C++	no	yes	libprotobuf	via DynamicMessage, slower
DynamicMessage (C++)	yes	no	libprotobuf	yes
upb	yes	optional	bundled C runtime	yes
nanopb	no	yes	none (embedded C)	no

Pick pbwire when the schema is only known at runtime, when one C core serves several host languages, or when libprotobuf must not be in the process. Pick generated code only when the schema never changes and you accept a codegen step: even then generated straight-line writes win only the smallest encode cells against dynamic dispatch — the Java and Kotlin bindings at 136 B (0.69x and 0.82x) — while the in-process pbw_ctx wins every tier of the core benchmark and the C struct codec wins every tier of its own.

Contributing

See CONTRIBUTING.md. Enable hooks once per clone:

./scripts/setup-git-hooks.sh

License

Apache-2.0.