Liking cljdoc? Tell your friends :D

Serialization

Usage

Serializers/deserializers can be passed in the factory function when creating a consumer or producer.

Your usage of serializer and options should always be symmetrical. That is, whatever you encode, you must be able to decode. Do not for example mix a nippy serializer and a fressian deserializer. This applies at the partition level, but it is recommended you also do not mix across partitions as your consumer will be unable to handle the results of an encoding it is not prepared to decode. Likewise, the same applies to a producer.

Alternatively, you may specify the serializer/deserializer via qualified class reference by using the Kafka config properties:

NameDescriptionExample
value.serializerserializes your values"org.apache.kafka.common.serialization.StringSerializer"
key.serializerserializes your keys"org.apache.kafka.common.serialization.LongSerializer"
value.deserializerdeserializes your values"org.apache.kafka.common.serialization.StringDeserializer"
key.deserializerdeserializes your keys"org.apache.kafka.common.serialization.LongDeserializer"

Serializers/Deserializers

Please create a work item if there is a serializer you need that is missing. One of the goals of Franzy is to provide comprehensive serialization solutions that can be added as libraries to minimize the impact of serializer related dependencies and development roadblocks.

The following serializers/deserializers are currently available or planned for your use:

NameSpeedCompressionEase of Use
EDNgreatminimal, but can composegreat
Stringgreatnonepoor, useful for keys or simple data only
Keywordgreatnonepoor, useful for keys or simple data only
Integergreatnonepoor, useful for keys or simple data only
Longgreatnonepoor, useful for keys or simple data only
Byte Arraygreatnone, but can composeworst, hard in-line code, but versatile
Simple EDNgood, great for small dataminimal, but can composegreat, but can lead to easy OOM error
JSONgoodnone/native jsongood, but requires settings/handlers for complex types/preservations of types
JSON Smileokgoodgood, but same as JSON with possible risk of Smile-issues
Nippyvery goodgreatgreat, probably the best current balance of speed and compression
Fressiangoodvery goodgood, but prefer nippy in most cases unless domain reasons
Debugpoor/dependsdependsok, not intended for production use, but can compose anything and pass back info
Transitgreatgoodgreat, but should match use case, otherwise prefer nippy
Avrogreatgreatpoor, a bit pedantic and requires up-front schema
UUID/SQUID (planned)greatnonepoor, useful for keys or simple data only
Gzip (planned)greatvery goodprefer nippy, unless specific domain reason

Available Serializers

  • EDN - built-in, good for Clojure data values
  • Simple EDN - built-in, good for small Clojure values
  • Integer - built-in, good for keys
  • Long - built-in, good for keys
  • String built-in, good for keys
  • Byte Array - built-in, good for values, use if you want to manually handle de/serialization for some strange reason
  • Keyword - built-in, good for keys
  • Debug - built-in, good for debugging, a shocker - can compose other serializers and log the output
  • Franzy-JSON - JSON Serialization with optional Smile support
  • Franzy-Nippy - Nippy serialization - highly recommended
  • Franzy-Fressian - Fressian serialization, especially useful for those integrating with Datomic
  • Franzy Transit - Transit with support for JSON, JSON-verbose, msgpack
  • Franzy Avro - Avro with support for EDN

Can you improve this documentation? These fine people already did:
One of many zeros & ymilky
Edit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close