kafka-clj.metadata — kafka-clj 4.0.3

All functions are concerned with retreiving metadata and offsets from kafka brokers part of this task requires blacklisting of brokers that do not response and retrying on connection errors

blacklisted? clj

(blacklisted? {:keys [driver] :as st} host-address)

close clj

(close {:keys [driver]})

connector clj

(connector bootstrap-brokers conf)

Creates a tcp pool and initiate for all brokers

get-cached-brokers clj

(get-cached-brokers metadata-connector)

(get-cached-brokers metadata-connector topic)

Return the current registered brokers in the metadata cache

get-cached-metadata clj

(get-cached-metadata metadata-connector topic)

(get-cached-metadata metadata-connector topic partition)

from the metadata connector, reads the metadata in the metadata-ref

Just calling metadata without a partition returns ;;[{:host "localhost", :port 50738, :isr [{:host "localhost", :port 50738}] specifying a partition returns the corresponding entry

get-metadata clj

(get-metadata {:keys [driver]} conf)

Return metadata in the form ;;validates metadata responses like {"abc" [{:host "localhost", :port 50738, :isr [{:host "localhost", :port 50738}], :id 0, :error-code 0}]}

META-RESP-SCHEMA clj

safe-nth clj

(safe-nth coll i)

send-recv-metadata-request clj

(send-recv-metadata-request conn conf)

Writes out a metadata request to the producer con

update-metadata! clj

(update-metadata! {:keys [brokers-ref metadata-ref] :as connector} conf)

Updates the borkers-ref, metadata-ref and add/remove hosts as per the metadata from the driver if meta is nil, any updates are skipped

write-metadata-request clj

(write-metadata-request buff
                        {:keys [correlation-id client-id]
                         :or {correlation-id 1 client-id "1"}})

see http://kafka.apache.org/protocol.html RequestMessage => ApiKey ApiVersion CorrelationId ClientId RequestMessage ApiKey => int16 ApiVersion => int16 CorrelationId => int32 ClientId => string MetadataRequest => [TopicName] TopicName => string