jackdaw.data — net.clojars.savya/jackdaw 1.3.1

This namespace aims to provide ->T, (datafy T), and data->T as a round-tripping of Katka's (client) record types.

Note that for some types, particularly Kafka's -Result types no ->T constructors are provided as there are no consumers within the Kafka API for these records they are merely packed results.

For compatibility with Clojure before 1.10.0, a datafy function is provided. On 1.10 or after, it simply defers to clojure.datafy/datafy but before 1.10 it acts as a backport thereof.

+broker-config-resource-type+ clj

+create-type-timestamp+ clj

A timestamp type associated with the timestamp being from the record's creation. That is, the record timestamp was user supplied.

+log-append-type-timestamp+ clj

A timestamp type associated with the timestamp having been generated by Kafka when the record was produced, not having been specified by the user when the record was created.

+no-type-timestamp+ clj

A timestamp type associated with... not having a timestamp type.

+topic-config-resource-type+ clj

+unknown-config-resource-type+ clj

->broker-resource clj

(->broker-resource name)

->ConfigEntry clj

(->ConfigEntry k v)

value can be a string else is a map where value is the :value key

->ConfigResource clj

(->ConfigResource type name)

->ConfigResourceType clj

(->ConfigResourceType o)

->ConsumerRecord clj

(->ConsumerRecord {:keys [:topic-name]}
                  partition
                  offset
                  ts
                  ts-type
                  key-size
                  value-size
                  key
                  value
                  headers)

Given unrolled ctor-style arguments create a Kafka ConsumerRecord.

Convenient for testing the consumer API and its helpers.

->OffsetAndTimestamp clj

(->OffsetAndTimestamp {:keys [offset timestamp]})

->ProducerRecord clj

(->ProducerRecord {:keys [topic-name]} value)

(->ProducerRecord {:keys [topic-name]} key value)

(->ProducerRecord {:keys [topic-name]} partition key value)

(->ProducerRecord {:keys [topic-name]} partition timestamp key value)

(->ProducerRecord {:keys [topic-name]} partition timestamp key value headers)

Given unrolled ctor-style arguments creates a Kafka ProducerRecord.

->RecordMetadata clj

(->RecordMetadata t partition offset timestamp key-size value-size)

(->RecordMetadata t
                  partition
                  base-offset
                  relative-offset
                  timestamp
                  key-size
                  value-size)

Given unrolled ctor-style arguments, create a Kafka RecordMetadata.

Note that as of KIP-31, Kafka actually only stores offsets relative to a message batch on the wire or on disk. In order to maintain the previous abstraction that there's a "offset" field which is absolute, an additional arity is provided which lets the user construct a record with a base offset and a relative offset of 0 so that the metadata's apparent offset is predictable.

->TimestampType clj

(->TimestampType kw)

Given a keyword being a datafied Kafka TimestampType, return the equivalent TimestampType instance.

->topic-resource clj

(->topic-resource name)

->TopicPartition clj

(->TopicPartition {:keys [:topic-name]} partition)

Given unrolled ctor-style arguments, create a Kafka TopicPartition.

as-OffsetAndTimestamp clj

(as-OffsetAndTimestamp ot)

as-TopicPartition clj

(as-TopicPartition o)

Config->data clj

(Config->data c)

ConfigEntry->data clj

(ConfigEntry->data e)

ConfigResource->data clj

(ConfigResource->data cr)

ConfigResourceType->data clj

(ConfigResourceType->data crt)

ConsumerRecord->data clj

(ConsumerRecord->data r)

datafy clj

(datafy x)

Attempts to return x as data.

datafy will return the value of #'clojure.core.protocols/datafy.

If the value has been transformed and the result supports metadata, :clojure.datafy/obj will be set on the metadata to the original value of x, and :clojure.datafy/class to the name of the class of x, as a symbol.

DescribeClusterResult->data clj

(DescribeClusterResult->data dcr)

map->Config clj

(map->Config m)

map->ConsumerRecord clj

(map->ConsumerRecord {:keys [:key :value :headers :partition :timestamp
                             :timestamp-type :offset :serialized-key-size
                             :serialized-value-size]
                      :as m})

Given a ::consumer-record, build an equivalent ConsumerRecord.

Inverts (datafy ^ConsumerRecord cr).

map->NewTopic clj

(map->NewTopic {:keys [:topic-name :partition-count :replication-factor
                       :topic-config]
                :as m})

map->OffsetAndTimestamp clj

(map->OffsetAndTimestamp m)

map->ProducerRecord clj

(map->ProducerRecord {:keys [topic-name key value headers partition timestamp]})

Given a ::producer-record build an equivalent ProducerRecord.

Inverts (datafy ^ProducerRecord r).

map->Properties clj

(map->Properties m)

Given a mapping of keywords to values, stringify the keys via #'clojure.walk/stringify-keys and return a Properties object. Scalar values (numbers, booleans) are also stringified: Properties.getProperty only returns String values, so an integer-valued config key would otherwise read back as nil (#311). Non-scalar values are passed through unchanged.

map->RecordMetadata clj

(map->RecordMetadata {:keys [:partition :timestamp :offset :serialized-key-size
                             :serialized-value-size]
                      :as m})

Given a ::record-metdata, build an equivalent RecordMetadata.

Inverts (datafy ^RecordMetadata rm).

map->TopicPartition clj

(map->TopicPartition {:keys [partition] :as m})

Given a topic-partition, build an equivalent TopicPartition.

Inverts (datafy ^TopicPartition tp).

Node->data clj

(Node->data node)

OffsetAndTimestamp->data clj

(OffsetAndTimestamp->data ots)

PartitionInfo->data clj

(PartitionInfo->data pi)

ProducerRecord->data clj

(ProducerRecord->data pr)

Properties->data clj

(Properties->data o)

Consume a Properties instance, keywordizing the keys and returning a Clojure mapping of the resulting keys to unmodified values.

RecordMetadata->data clj

(RecordMetadata->data rm)

TestRecord->data clj

(TestRecord->data tr)

TimestampType->data clj

(TimestampType->data tt)

TopicDescription->data clj

(TopicDescription->data td)

TopicPartition->data clj

(TopicPartition->data tp)

TopicPartitionInfo->data clj

(TopicPartitionInfo->data tpi)