Serialization

Usage

Serializers/deserializers can be passed in the factory function when creating a consumer or producer.

Your usage of serializer and options should always be symmetrical. That is, whatever you encode, you must be able to decode. Do not for example mix a nippy serializer and a fressian deserializer. This applies at the partition level, but it is recommended you also do not mix across partitions as your consumer will be unable to handle the results of an encoding it is not prepared to decode. Likewise, the same applies to a producer.

Alternatively, you may specify the serializer/deserializer via qualified class reference by using the Kafka config properties:

Name	Description	Example
value.serializer	serializes your values	"org.apache.kafka.common.serialization.StringSerializer"
key.serializer	serializes your keys	"org.apache.kafka.common.serialization.LongSerializer"
value.deserializer	deserializes your values	"org.apache.kafka.common.serialization.StringDeserializer"
key.deserializer	deserializes your keys	"org.apache.kafka.common.serialization.LongDeserializer"

Serializers/Deserializers

Please create a work item if there is a serializer you need that is missing. One of the goals of Franzy is to provide comprehensive serialization solutions that can be added as libraries to minimize the impact of serializer related dependencies and development roadblocks.

The following serializers/deserializers are currently available or planned for your use:

Name	Speed	Compression	Ease of Use
EDN	great	minimal, but can compose	great
String	great	none	poor, useful for keys or simple data only
Keyword	great	none	poor, useful for keys or simple data only
Integer	great	none	poor, useful for keys or simple data only
Long	great	none	poor, useful for keys or simple data only
Byte Array	great	none, but can compose	worst, hard in-line code, but versatile
Simple EDN	good, great for small data	minimal, but can compose	great, but can lead to easy OOM error
JSON	good	none/native json	good, but requires settings/handlers for complex types/preservations of types
JSON Smile	ok	good	good, but same as JSON with possible risk of Smile-issues
Nippy	very good	great	great, probably the best current balance of speed and compression
Fressian	good	very good	good, but prefer nippy in most cases unless domain reasons
Debug	poor/depends	depends	ok, not intended for production use, but can compose anything and pass back info
Transit	great	good	great, but should match use case, otherwise prefer nippy
Avro	great	great	poor, a bit pedantic and requires up-front schema
UUID/SQUID (planned)	great	none	poor, useful for keys or simple data only
Gzip (planned)	great	very good	prefer nippy, unless specific domain reason

Available Serializers

• EDN - built-in, good for Clojure data values
• Simple EDN - built-in, good for small Clojure values
• Integer - built-in, good for keys
• Long - built-in, good for keys
• String built-in, good for keys
• Byte Array - built-in, good for values, use if you want to manually handle de/serialization for some strange reason
• Keyword - built-in, good for keys
• Debug - built-in, good for debugging, a shocker - can compose other serializers and log the output
• Franzy-JSON - JSON Serialization with optional Smile support
• Franzy-Nippy - Nippy serialization - highly recommended
• Franzy-Fressian - Fressian serialization, especially useful for those integrating with Datomic
• Franzy Transit - Transit with support for JSON, JSON-verbose, msgpack
• Franzy Avro - Avro with support for EDN