semantic-csv.transducers — semantic-csv 0.2.1-alpha1

Transducers API namespace

->boolean clj/s≠

clj

(->boolean x)

(->boolean {:keys [nil-fill]} x)

Translate to boolean from string or other numeric.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->double clj/s≠

clj

(->double x)

(->double {:keys [nil-fill]} x)

Translate to double from string or other numeric.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->float clj/s≠

clj

(->float x)

(->float {:keys [nil-fill]} x)

Translate to float from string or other numeric.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->idiomatic-keyword clj/s≠

clj

(->idiomatic-keyword x)

Takes a string, replacing consecutive underscores and spaces with a single dash(-), then returns a keyword based on the transformed string.

cljs

->int clj/s≠

clj

(->int x)

(->int {:keys [nil-fill]} x)

Translate to int from string or other numeric. If string represents a non integer value, it will be rounded down to the nearest int.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->long clj/s≠

clj

(->long x)

(->long {:keys [nil-fill]} x)

Translate to long from string or other numeric. If string represents a non integer value, will be rounded down to the nearest long.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

batch clj/s

(batch n)

Returns a transducer that will return a sequence of row batches, where the batch size is n.

cast-with clj/s

(cast-with cast-fns)

(cast-with cast-fns {:as opts :keys [except-first exception-handler only]})

Returns a transducer that casts the vals of each row according to cast-fns, which must either be a map of column-name -> casting-fn or a single casting function to be applied towards all columns. Options include:

• :except-first - Leaves the first row unaltered; useful for preserving header row.
• :exception-handler - If cast-fn raises an exception, this function will be called with args colname, value, and the result used as the parse value.
• :only - Only cast the specified column(s); can be either a single column name, or a vector of them.

mappify clj/s

(mappify)

(mappify {:as opts :keys [keyify transform-header header] :or {keyify true}})

Returns a transducer that transforms row vectors into maps, where keys are taken as the first row or as specified via :header option. Options include:

• :keyify - bool; specify whether header/column names should be turned into keywords (default: true).
• :header - specify the header to use for map keys, preventing first row of data from being consumed as header.
• :transform-header - specify a transformation function for each header key (ignored if :header or :keyify is specified).

parse-and-process clj

(parse-and-process csv-readable
                   &
                   {:keys [parser-opts] :or {parser-opts {}} :as opts})

This is a convenience function for reading a csv file using clojure-csv and passing it through process with the given set of options (specified last as kw args, in contrast with our other processing functions). Note that :parser-opts can be specified and will be passed along to clojure-csv/parse-csv

process clj/s≠

clj

(process)

(process {:keys [mappify keyify header transform-header remove-comments
                 comment-re comment-char structs cast-fns cast-exception-handler
                 cast-only]
          :or {mappify true keyify true remove-comments true comment-re #"^\#"}
          :as opts})

cljs

(process)

(process {:keys [mappify keyify header transform-header remove-comments
                 comment-re comment-char structs cast-fns cast-exception-handler
                 cast-only]
          :or {mappify true keyify true remove-comments true comment-re #"^\#"}
          :as opts})

Returns a transducers that composes the most frequently used input processing capabilities, and is controlled by an opts hash with opinionated defaults:

• :mappify - bool; transform rows from vectors into maps using mappify.
• :keyify - bool; specify whether header/column names should be turned into keywords (default: true).
• :transform-header - specify a transformation function for each header key (ignored if :header or :keyify is specified).
• :header - specify header to be used in mappify; as per mappify, first row will not be consumed as header
• :structs - bool; use structify insead of mappify
• :remove-comments - bool; remove comment lines, as specified by :comment-re or :comment-char. Also removes empty lines. Defaults to true.
• :comment-re - specify a regular expression to use for commenting out lines.
• :comment-char - specify a comment character to use for filtering out comments; overrides comment-re.
• :cast-fns - optional map of colname | index -> cast-fn; row maps will have the values as output by the assigned cast-fn.
• :cast-exception-handler - If cast-fn raises an exception, this function will be called with args colname, value, and the result used as the parse value.
• :cast-only - Only cast the specified column(s); can be either a single column name, or a vector of them.

remove-comments clj/s

(remove-comments)

(remove-comments {:as opts :keys [comment-re comment-char]})

Returns a transducer that removes rows starting with a comment character (by default, #). Operates by checking whether the first item of every row in the collection matches a comment pattern. Also removes empty lines. Options include:

• :comment-re - Specify a custom regular expression for determining which lines are commented out.
• :comment-char - Checks for lines lines starting with this char.

Note: this function only works (or makes sense) with rows that are vectors, and so should always be used before mappify.

slurp-csv clj

(slurp-csv csv-filename & {:as opts})

This convenience function let's you parse-and-process csv data given a csv filename. Note that it is not lazy, and must read in all data so the file handle can be closed.

spit-csv clj

(spit-csv file rows)

(spit-csv file
          {:keys [batch-size cast-fns writer-opts header prepend-header]
           :or {batch-size 20 prepend-header true}
           :as opts}
          rows)

Convenience function for spitting out CSV data to a file using clojure-csv.

• file - Can be either a filename string, or a file handle.
• opts - Optional hash of settings.
• rows - Can be a sequence of either maps or vectors; if the former, vectorize will be called on the input with :header argument specifiable through opts.

The Options hash can have the following mappings:

• :batch-size - How many rows to format and write at a time?
• :cast-fns - Formatter(s) to be run on row values. As with cast-with function, can be either a map of column-name -> cast-fn, or a single function to be applied to all values. Note that str is called on all values just before writing regardless of :cast-fns.
• :writer-opts - Options hash to be passed along to clojure-csv.core/write-csv.
• :header - Header to be passed along to vectorize, if necessary.
• :prepend-header - Should the header be prepended to the rows written if vectorize is called?

structify clj

(structify)

(structify {:as opts :keys [keyify transform-header header] :or {keyify true}})

Takes an optional map of input options and returns a transducer. The transducer transforms row vectors into structs, where keys are taken as the first row or as specified via :header option. This is analogous to mappify. Options:

• :keyify - bool; specify whether header/column names should be turned into keywords (default: true).
• :header - specify the header to use for map keys, preventing first row of data from being consumed as header.
• :transform-header - specify a transformation function for each header key (ignored if :header or :keyify is specified).

vectorize clj/s

(vectorize)

(vectorize {:keys [header prepend-header format-header]
            :or {prepend-header true format-header impl/stringify-keyword}})

Returns a transducer that transforms maps into vectors. Options include:

• :header - The header to be used. If not specified, this defaults to (-> rows first keys). Only values corresponding to the specified header will be included in the output, and will be included in the order corresponding to this argument.
• :prepend-header - Defaults to true, and controls whether the :header vector should be prepended to the output sequence.
• :format-header - If specified, this function will be called on each element of the :header vector, and the result prepended to the output sequence. The default behaviour is to leave strings alone but stringify keyword names such that the : is removed from their string representation. Passing a falsey value will leave the header unaltered in the output.