prpr.stream.chunk

Liking cljdoc? Tell your friends :D

All platforms.

dechunk
default-chunk-size
make-chunker-xform
rechunk
should-finish-chunk?
stream-chunk-builder

dechunk^clj/s

(dechunk s)

given a stream of mixed unchunked-values and chunks return a stream of [::unchunked|::chunk-start|::chunk|::chunk-end val]

given a stream of mixed unchunked-values and chunks
return a stream of
[::unchunked|::chunk-start|::chunk|::chunk-end val]

source raw docstring

default-chunk-size^clj/s

source

make-chunker-xform^clj/s

(make-chunker-xform target-chunk-size)

(make-chunker-xform target-chunk-size partition-by-fn)

return a transducer which builds chunks from a stream, optionally partition-by the stream ensuring that partitions never span chunk boundaries

NOTE that no timeout is possible with a transducer

target-chunk-size : will wrap a chunk when this size is exceeded, or as soon as possible afterwards (if a chunk is received, or partition-by is given)
partition-by-fn : also partition-by the stream with this fn and ensure partitions never cross chunk boundaries

return a transducer which builds chunks from a stream, optionally
partition-by the stream ensuring that partitions never span
chunk boundaries

NOTE that no timeout is possible with a transducer

- target-chunk-size : will wrap a chunk when this size is exceeded,
    or as soon as possible afterwards (if a chunk is received, or
    partition-by is given)
- partition-by-fn : also partition-by the stream with this fn and
    ensure partitions never cross chunk boundaries

source raw docstring

rechunk^clj/s

(rechunk s)

given a stream of [::unchunked|::chunk-start|::chunk|::chunk-end val] return a stream of unchunked values and chunks

given a stream of
[::unchunked|::chunk-start|::chunk|::chunk-end val]
return a stream of unchunked values and chunks

source raw docstring

should-finish-chunk?^clj/s

(should-finish-chunk? chunk-builder
                      target-chunk-size
                      partition-by-fn
                      next-value)

true if building a chunk and:

(nil? partition-by) and (>= chunk-size target-chunk-size)
(some? partition-by-fn) and (>= chunk-size target-chunk-size) and (not= (partition-by-fn (last chunk-state) (partition-by next-value)))

true if building a chunk and:

- (nil? partition-by) and (>= chunk-size target-chunk-size)
- (some? partition-by-fn)
  and (>= chunk-size target-chunk-size)
  and (not= (partition-by-fn (last chunk-state) (partition-by next-value)))

source raw docstring