decoder

;; A reducible that opens, decodes, and closes
(decoder/make open decode done? close)

A decoder is a stateful reducible with four hooks: open, decode, done?, close. A decoder supplies data to transductions so you can, for example, open a CSV file with a decoder and efficiently process data from it with transducers. This is its primary purpose: supplying data to transductions through I/O.

How it works

open returns a stateful, mutable object (e.g. an instance of java.io.InputStream). Let's call this return value in. When asked to reduce, the decoder will call (decode in) and produce some value, which we'll call value. Then it calls (done? value) if the result is truthy, then the reduction ends, otherwise the decoder adds value to the accumulator and starts the whole process over again by calling (decode in).

When the reduction process terminates (early or otherwise) the decoder calls (close in). If close is not supplied, then it defaults to java.lang.AutoCloseable/.close which works for most I/O scenarios.

Comparison to clojure.core/iteration

Unlike clojure.core/iteration, a decoder repeatedly calls decode with the return value from open. open should return a stateful, mutable object that produces a new value every time it is given to decode, there is no succession of continuation tokens. This is simpler to think about and use in common situations like java.io streams and readers.

A decoder also cleans up resources when its reduction completes (early or otherwise). This makes it suitable for working with things like database connection, file descriptors, etc. If an error occurs during the reduction, or if the reduction returns early (i.e. someone returns a reduced), there is still an opportunity to release resources when the decoder calls close.

Example

We could create a decoder to read Clojure data (note we do not need to specify close since java.io.PushbackReader implements java.lang.AutoCloseable):

(require '[systems.thoughtfull.decoder :as decoder])

(def open (comp java.io.PushbackReader/new java.io.StringReader/new))
(def decode #(read % false ::eof))
(def done? #{::eof})

(def xf (comp (filter odd?) (map inc)))
(def clojure-decoder (decoder/make open decode done?))

(println (into [] xf (clojure-decoder "1 2 3"))) ;;=> [2 4]

License

Copyright © technosophist

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

This Source Code Form is "Incompatible With Secondary Licenses", as defined by the Mozilla Public License, v. 2.0.