jackdaw.streams.protocols — fundingcircle/jackdaw 0.7.2

Kafka streams protocols.

IGlobalKTable clj protocol

global-ktable* clj

(global-ktable* globalktable)

Returns the underlying GlobalKTable

IKGroupedBase clj protocol

Methods shared between IKGroupedTable and IKGroupedStream.

aggregate clj

(aggregate kgrouped initializer-fn aggregator-fn)

(aggregate kgrouped initializer-fn aggregator-fn subtractor-fn-or-topic-config)

(aggregate kgrouped
           initializer-fn
           adder-fn
           subtractor-or-merger-fn
           topic-config)

Aggregates values by key into a new KTable.

count clj

(count kgrouped)

(count kgrouped topic-config)

Counts the number of records by key into a new KTable.

reduce clj

(reduce kgrouped reducer-fn)

(reduce kgrouped reducer-fn subtractor-fn-or-topic-config)

(reduce kgrouped adder-fn subtractor-fn topic-config)

Combines values of a stream by key into a new KTable.

IKGroupedStream clj protocol

KGroupedStream is an abstraction of a grouped stream.

kgroupedstream* clj

(kgroupedstream* kgroupedstream)

Returns the underlying KGroupedStream object.

windowed-by-session clj

(windowed-by-session kgroupedstream window)

windowed-by-time clj

(windowed-by-time kgroupedstream window)

IKGroupedTable clj protocol

KGroupedTable is an abstraction of a grouped changelog stream.

kgroupedtable* clj

(kgroupedtable* kgroupedtable)

Returns the underlying KGroupedTable object.

IKStream clj protocol

A KStream is an abstraction of a stream of key-value pairs.

left-join-global clj

(left-join-global kstream global-ktable kv-mapper joiner)

transform-values clj

(transform-values kstream value-transformer-supplier-fn)

(transform-values kstream value-transformer-supplier-fn state-store-names)

Creates a KStream that consists of the results of applying the transformer to each value in the input stream.

branch clj

(branch kstream predicate-fns)

Returns a list of KStreams, one for each of the predicate-fns provided.

flat-map clj

(flat-map kstream key-value-mapper-fn)

Creates a KStream that will consist of the concatenation of messages returned by calling key-value-mapper-fn on each key/value pair in the input stream.

map clj

(map kstream key-value-mapper-fn)

Creates a KStream that consists of the result of applying key-value-mapper-fn to each key/value pair in the input stream.

print! clj

(print! kstream)

Prints the elements of the stream to out.

flat-map-values clj

(flat-map-values kstream value-mapper-fn)

Creates a KStream that will consist of the concatenation of the values returned by calling value-mapper-fn on each value in the input stream.

process! clj

(process! kstream processor-fn state-store-names)

Applies processor-fn to each item in the input stream.

to! clj

(to! kstream topic-config)

Materializes a stream to a topic.

group-by-key clj

(group-by-key kstream)

(group-by-key kstream topic-config)

Groups records with the same key into a KGroupedStream.

transform clj

(transform kstream transformer-supplier-fn)

(transform kstream transformer-supplier-fn state-store-names)

Creates a KStream that consists of the results of applying the transformer to each key/value in the input stream.

through clj

(through kstream topic-config)

Materializes a stream to a topic, and returns a new KStream that will consume messages from the topic. Messages in the new topic will be partitioned based on the output of the optional partition function that represents StreamPartitioner class

for-each! clj

(for-each! kstream foreach-fn)

Performs an action on each element of KStream.

outer-join-windowed clj

(outer-join-windowed kstream other-kstream value-joiner-fn windows)

(outer-join-windowed kstream
                     other-kstream
                     value-joiner-fn
                     windows
                     this-topic-config
                     other-topic-config)

Combines the values of two streams that share the same key using a windowed outer join.

left-join-windowed clj

(left-join-windowed kstream other-kstream value-joiner-fn windows)

(left-join-windowed kstream
                    other-kstream
                    value-joiner-fn
                    windows
                    this-topic-config
                    other-topic-config)

Combines the values of two streams that share the same key using a windowed left join.

kstream* clj

(kstream* kstream)

Returns the underlying KStream object.

join-global clj

(join-global kstream global-ktable kv-mapper joiner)

select-key clj

(select-key kstream select-key-value-mapper-fn)

Create a new key from the current key and value.

select-key-value-mapper-fn should be a function that takes a key-value pair, and returns the value of the new key. Here is example multiplies each key by 10:

(fn [[k v]] (* 10 k))

merge clj

(merge kstream other)

Creates a KStream that has the records from both streams.

join-windowed clj

(join-windowed kstream other-kstream value-joiner-fn windows)

(join-windowed kstream
               other-kstream
               value-joiner-fn
               windows
               this-topic-config
               other-topic-config)

Combines the values of two streams that share the same key using a windowed inner join.

IKStreamBase clj protocol

Methods common to KStream and KTable.

filter clj

(filter kstream-or-ktable predicate-fn)

Creates a KStream that consists of all elements that satisfy a predicate.

filter-not clj

(filter-not kstream-or-ktable predicate-fn)

Creates a KStream that consists of all elements that do not satisfy a predicate.

group-by clj

(group-by ktable-or-ktable key-value-mapper-fn)

(group-by ktable-or-ktable key-value-mapper-fn topic-config)

Groups the records of this KStream/KTable using the key-value-mapper-fn.

left-join clj

(left-join kstream-or-ktable ktable value-joiner-fn)

(left-join kstream-or-ktable
           ktable
           value-joiner-fn
           this-topic-config
           other-topic-config)

Creates a KStream from the result of calling value-joiner-fn with each element in the KStream and the value in the KTable with the same key.

map-values clj

(map-values kstream-or-ktable value-mapper-fn)

Creates a KStream that is the result of calling value-mapper-fn on each element of the input stream.

peek clj

(peek kstream-or-ktable peek-fn)

Performs peek-fn on each element of the input stream.

write-as-text! clj

(write-as-text! kstream-or-ktable file-path)

(write-as-text! kstream-or-ktable file-path topic-config)

Writes the elements of a stream to a file at the given path.

IKTable clj protocol

A Ktable is an abstraction of a changlog stream.

join clj

(join ktable other-ktable value-joiner-fn)

Combines the values of the two KTables that share the same key using an inner join.

ktable* clj

(ktable* ktable)

Returns the underlying KTable object.

outer-join clj

(outer-join ktable other-ktable value-joiner-fn)

Combines the values of two KTables that share the same key using an outer join.

suppress clj

(suppress ktable {:keys [max-records max-bytes until-time-limit-ms]})

Suppress some updates from this changelog stream. You can either specify max-records or max-bytes. If an empty map is passed, the suppress will be unbounded. If until-time-limit-ms is set, this will override the TimeWindow interval. Note that when relying on the configured TimeWindow the default grace period is 24h - window-size.

to-kstream clj

(to-kstream ktable)

(to-kstream ktable key-value-mapper-fn)

Converts a KTable to a KStream.

ISessionWindowedKStream clj protocol

ISessionWindowedKStream is an abstraction of a session windowed stream.

session-windowed-kstream* clj

(session-windowed-kstream* ksession-windowed-kstream)

Returns the underlying SessionWindowedKStream object.

IStreamsBuilder clj protocol

global-ktable clj

(global-ktable topology-builder topic-config)

Creates a GlobalKTable that will consist of data from the specified topic.

kstream clj

(kstream topology-builder topic-config)

(kstream topology-builder topic-config topic-pattern)

Creates a KStream that will consume messages from the specified topic.

kstreams clj

(kstreams topology-builder topic-configs)

Creates a KStream that will consume messages from the specified topics.

ktable clj

(ktable topology-builder topic-config)

(ktable topology-builder topic-config store-name)

Creates a KTable that will consist of data from the specified topic.

source-topics clj

(source-topics topology-builder)

Gets the names of source topics for the topology.

streams-builder* clj

(streams-builder* streams-builder)

Returns the underlying KStreamBuilder.

ITimeWindowedKStream clj protocol

ITimeWindowedKStream is an abstraction of a time windowed stream.

time-windowed-kstream* clj

(time-windowed-kstream* ktime-windowed-kstream)

Returns the underlying TimeWindowedKStream object.