semantic-csv.core — semantic-csv 0.2.1-alpha1

Core API namespace

->boolean clj/s≠

clj

(->boolean x)

(->boolean {:keys [nil-fill]} x)

Translate to boolean from string or other numeric.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->double clj/s≠

clj

(->double x)

(->double {:keys [nil-fill]} x)

Translate to double from string or other numeric.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->float clj/s≠

clj

(->float x)

(->float {:keys [nil-fill]} x)

Translate to float from string or other numeric.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->idiomatic-keyword clj/s≠

clj

(->idiomatic-keyword x)

Takes a string, replacing consecutive underscores and spaces with a single dash(-), then returns a keyword based on the transformed string.

cljs

->int clj/s≠

clj

(->int x)

(->int {:keys [nil-fill]} x)

Translate to int from string or other numeric. If string represents a non integer value, it will be rounded down to the nearest int.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

->long clj/s≠

clj

(->long x)

(->long {:keys [nil-fill]} x)

Translate to long from string or other numeric. If string represents a non integer value, will be rounded down to the nearest long.

An opts map can be specified as the first arguments with the following options:

• :nil-fill - return this when input is empty/nil.

cljs

batch clj/s

(batch n rows)

Takes sequence of items and returns a sequence of batches of items from the original sequence, at most n long.

cast-with clj/s

(cast-with cast-fns rows)

(cast-with cast-fns opts rows)

Casts the vals of each row according to cast-fns, which must either be a map of column-name -> casting-fn or a single casting function to be applied towards all columns. Additionally, an opts map can be used to specify:

• :except-first - Leaves the first row unaltered; useful for preserving header row.
• :exception-handler - If cast-fn raises an exception, this function will be called with args colname, value, and the result used as the parse value.
• :only - Only cast the specified column(s); can be either a single column name, or a vector of them.

except-first clj/s macro

(except-first & forms-and-data)

Takes any number of forms and a final data argument. Threads the data through the forms, as though with ->>, except that the first item in data remains unaltered. This is intended to operate within the context of an actual ->> threading macro for processing, where you might want to leave a header column unmodified by your processing functions.

mappify clj/s

(mappify rows)

(mappify
  {:keys [keyify transform-header header structs] :or {keyify true} :as opts}
  rows)

Takes a sequence of row vectors, as commonly produced by csv parsing libraries, and returns a sequence of maps. By default, the first row vector will be interpreted as a header, and used as the keys for the maps. However, this and other behaviour are customizable via an optional opts map with the following options:

• :keyify - bool; specify whether header/column names should be turned into keywords (default: true). If :transform-header is present, this option will be ignored.
• :transform-header - A function that transforms the header/column names for each column. This takes precedence over keyify and should be a function that takes a string.
• :header - specify the header to use for map keys, preventing first row of data from being consumed as header.
• :structs - bool; use structs instead of hash-maps or array-maps, for performance boost (default: false).

parse-and-process clj

(parse-and-process csv-readable
                   &
                   {:keys [parser-opts] :or {parser-opts {}} :as opts})

This is a convenience function for reading a csv file using clojure/data.csv and passing it through process with the given set of options (specified last as kw_args, in contrast with our other processing functions). Note that :parser-opts can be specified and will be passed along to clojure-csv/parse-csv

process clj/s

(process rows)

(process opts rows)

This function wraps together the most frequently used input processing capabilities, controlled by an opts hash with opinionated defaults:

• :mappify - bool; transform rows from vectors into maps using mappify.
• :keyify - bool; specify whether header/column names should be turned into keywords (default: true).
• :header - specify header to be used in mappify; as per mappify, first row will not be consumed as header
• :structs - bool; use structs instead of array-maps or hash-maps in mappify.
• :remove-comments - bool; remove comment lines, as specified by :comment-re or :comment-char. Also removes empty lines. Defaults to true.
• :comment-re - specify a regular expression to use for commenting out lines.
• :comment-char - specify a comment character to use for filtering out comments; overrides comment-re.
• :cast-fns - optional map of colname | index -> cast-fn; row maps will have the values as output by the assigned cast-fn.
• :cast-exception-handler - If cast-fn raises an exception, this function will be called with args colname, value, and the result used as the parse value.
• :cast-only - Only cast the specified column(s); can be either a single column name, or a vector of them.

remove-comments clj/s

(remove-comments rows)

(remove-comments opts rows)

Removes rows which start with a comment character (by default, #). Operates by checking whether the first item of every row in the collection matches a comment pattern. Also removes empty lines. Options include:

• :comment-re - Specify a custom regular expression for determining which lines are commented out.
• :comment-char - Checks for lines lines starting with this char.

Note: this function only works with rows that are vectors, and so should always be used before mappify.

slurp-csv clj

(slurp-csv csv-filename & {:as opts})

This convenience function let's you parse-and-process csv data given a csv filename. Note that it is not lazy, and must read in all data so the file handle can be closed.

spit-csv clj

(spit-csv file rows)

(spit-csv file opts rows)

Convenience function for spitting out CSV data to a file using clojure-csv.

• file - Can be either a filename string, or a file handle.
• opts - Optional hash of settings.
• rows - Can be a sequence of either maps or vectors; if the former, vectorize will be called on the input with :header argument specifiable through opts.

The Options hash can have the following mappings:

• :batch-size - How many rows to format and write at a time?
• :cast-fns - Formatter(s) to be run on row values. As with cast-with function, can be either a map of column-name -> cast-fn, or a single function to be applied to all values. Note that str is called on all values just before writing regardless of :cast-fns.
• :writer-opts - Options hash to be passed along to clojure-csv.core/write-csv.
• :header - Header to be passed along to vectorize, if necessary.
• :prepend-header - Should the header be prepended to the rows written if vectorize is called?

vectorize clj/s

(vectorize rows)

(vectorize opts rows)

Take a sequence of maps, and transform them into a sequence of vectors. Options:

• :header - The header to be used. If not specified, this defaults to (-> rows first keys). Only values corresponding to the specified header will be included in the output, and will be included in the order corresponding to this argument.
• :prepend-header - Defaults to true, and controls whether the :header vector should be prepended to the output sequence.
• :format-header - If specified, this function will be called on each element of the :header vector, and the result prepended to the output sequence. The default behaviour is to leave strings alone but stringify keyword names such that the : is removed from their string representation. Passing a falsey value will leave the header unaltered in the output.