szew.io.util — szew/io 0.4.0

Useful data mangling functions.

bespoke-header clj

(bespoke-header a-map)

(bespoke-header base-header a-map)

(bespoke-header base-header strict? a-map)

Accepts vector and a map, returns header vector derived from both.

Header consists of base-header and remaining keys from the map, sorted.

If strict: base-header must be a subset of keys in the map, otherwise ExceptionInfo. If not strict then just 'do your best'.

If only given the map - returns vector of its sorted keys.

Why?

You want some key columns first, while keeping all the data.

How?

(bespoke-header [:x :y :z] {:state :resting :x 0 :y 0 :z 0}) => [:x :y :z :state] (bespoke-header [:x :y :z :cycle] {:state :resting :x 0 :y 0 :z 0}) => [:x :y :z :cycle :state] (bespoke-header [:x :y :z :cycle] true {:state :resting :x 0 :y 0 :z 0}) => clojure.lang.ExceptionInfo: Missing keys in header?

deep-sort clj

(deep-sort coll-of-colls)

Takes nested collection, returns it with sorted maps and sets.

Keeps meta.

Note: Sorting requires comparable values. Non-compliance means no order. Other containers are passed through.

Why?

Sometimes you want to be able to just look at the map and know if a key is there. Ordering keys helps most of the time.

How?

(deep-sort {100 :a, 90 {10 :maybe, 9 {:yes :no}}, 80 :b}) => {80 :b, 90 {9 {:yes :no}, 10 :maybe}, 100 :a}

default-parallel-n clj

(default-parallel-n)

Return advised parallel workload. Logical processor count + 2.

Why?

Sometimes you just want the answer.

How?

(default-parallel-n) => 10

field-unpacker clj

(field-unpacker delimiter field)

See if field contains a str/char delimiter, if so -- tries to parse as CSV.

Why?

Sometimes your fields have fields. (So many fields, fields for days.)

How?

(field-unpacker , "ala,ma,kota") => ["ala", "ma", "kota"]

fixed-width-split clj

(fixed-width-split slices)

Takes vector of slice sizes, returns function that will cut String.

Why?

Because somebody thought fixed width data records are a good thing.

How?

((fixed-width-split [4 3 4]) "Ala ma Kota.") => ["Ala " "ma " "Kota"]

friendlify clj

(friendlify a-something)

Take Clojure object and try to make a pretty String from its Class.

Why?

Display function name at runtime, but similar to how it looks in code.

How?

(friendlify friendlify) => "szew.io.util/friendlify"

getter clj

(getter a-key)

(getter a-key default)

Takes key and default value, returns a function that gets it.

Why?

Builtin get takes collection as first argument, this is the other way around.

How?

(mapv (getter :yolo :no) [{:yolo :yes} {:wat? :wat}]) => [:yes :no] (meta (getter :yolo :no)) => {:key :yolo, :default :no} (mapv (getter 0 :no) [[:yes] []]) => [:yes :no] (meta (getter 1 :no)) => {:key 1, :default :no}

juxt-map clj

(juxt-map & keys-fns)

Takes keys and values, get juxt map of keys-values. Hint: zipmap over juxt.

Why?

Maps are bees knees and juxt is just cool.

How?

((juxt-map :+ inc :- dec := identity) 2) => {:+ 3, :- 1, := 2}

map->vec clj

(map->vec header mapped)

(map->vec defaults header mapped)

Takes header vector and map mapped. Returns vector with meta.

The defaults map is used if key is missing from mapped.

Why?

Some formats serialize naturally from a vector. Uniform vectors are nice.

How?

(map->vec [:k1 :k2] {:k1 1, :k2 2}) => [1 2] (map->vec {:kx 100} [:k1 :k2 :kx] {:k1 1, :k2 2}) => [1 2 100] (meta (map->vec {:kx 100} [:k1 :k2 :kx] {:k1 1 :k2 2})) => {:defaults {:kx 100}, :header [:k1 :k2 :kx], :mapped {:k1 1, :k2 2}}

maps->vecs clj

(maps->vecs a-seq)

(maps->vecs head tail)

(maps->vecs defaults head tail)

Takes header vector and maps collection tail, returns sequence of vectors.

The defaults map is used if key is missing from tail map.

Why?

Sometimes you just need your data as uniform vectors instead of maps.

How?

(maps->vecs [[:k1 :k2] {:k1 1, :k2 2}]) => ([1 2]) (maps->vecs [:k1 :k2] [{:k1 1, :k2 2}]) => ([1 2]) (maps->vecs {:kx 100} [:k1 :k2 :kx] [{:k1 1, :k2 2}]) => ([1 2 100]) (maps->vecs [:k1 :k2] [{:k1 1, :k2 2} {:k1 1, :k2 0}]) => ([1 2] [1 0])

maps-maker clj

(maps-maker)

(maps-maker head tail)

Create function of collection of vectors using two callables: head and tail.

By default head is first row as kebab-case-keyword vector and tail is rest.

Why?

Ingest spreadsheet like grid data, get a sequence of sane-ish maps.

How?

((maps-maker) [["X" "Y"] [0 0] [1 1] [10 10]]) => ({:x 0 :y 0} {:x 1 :y 1} {:x 10 :y 10})

pp-filter clj

(pp-filter n p)

(pp-filter n p coll)

Parametrized parallel filter, with max of n threads at once.

Accepts number of threads n, predicate p and a collection. If no collection is provided it produces a stateful transducer. Uses futures.

Why?

The real question is: why no pfilter!?

How?

(vec (filter odd? [1 2 3])) => [1 3] (vec (pp-filter 1 odd? [1 2 3])) => [1 3]

pp-keep clj

(pp-keep n f)

(pp-keep n f coll)

Parametrized parallel keep, with max of n threads at once.

Accepts number of threads n, function f and a collection. If no collection is provided it produces a stateful transducer. Uses futures.

Why?

Free once pp-map is defined.

How?

(vec (keep {:x 1} [:x :y :z])) => [1] (vec (pp-keep 1 {:x 1} [:x :y :z])) => [1]

pp-keep-indexed clj

(pp-keep-indexed n f)

(pp-keep-indexed n f coll)

Parametrized parallel keep-indexed, with max of n threads at once.

Accepts number of threads n, function f and a collection. If no collection is provided it produces a stateful transducer. Uses futures.

Why?

Free once pp-map-indexed is defined.

How?

(vec (keep-indexed (fn [_ k] (get {:x 1} k)) [:x :y :z])) => [1] (vec (pp-keep-indexed 1 (fn [_ k] (get {:x 1} k)) [:x :y :z])) => [1]

pp-map clj

(pp-map n f)

(pp-map n f coll)

(pp-map n f coll & colls)

Parametrized parallel map, with max of n threads at once.

Accepts number of threads n, function f and collections. If no collections are provided it produces a stateful transducer. Uses futures.

Why?

Better pmap for multicore world.

How?

(vec (map inc [1 2 3])) => [2 3 4] (vec (pp-map 1 inc [1 2 3])) => [2 3 4]

pp-map-indexed clj

(pp-map-indexed n f)

(pp-map-indexed n f coll)

Parametrized parallel map-indexed, with max of n threads at once.

Accepts number of threads n, function f and collections. If no collections are provided it produces a stateful transducer. Uses futures.

Why?

You sometimes need to compose it, that's why.

How?

(vec (map-indexed (fn [i k] [i (get {:x 1} k)]) [:x :y :z])) => [[0 1] [1 nil] [2 nil]] (vec (pp-map-indexed 1 (fn [i k] [i (get {:x 1} k)]) [:x :y :z])) => [[0 1] [1 nil] [2 nil]]

pp-mapcat clj

(pp-mapcat n f)

(pp-mapcat n f & colls)

Parametrized parallel mapcat, with max of n threads at once for mapping.

Accepts number of threads n, function f and a collection. If no collection is provided it produces a stateful transducer. Uses futures.

Why?

Free once ppmap is defined.

How?

(mapcat (juxt dec identity inc) [1 2 3 4]) => (0 1 2 1 2 3 2 3 4 3 4 5) (pp-mapcat 4 (juxt dec identity inc) [1 2 3 4]) => (0 1 2 1 2 3 2 3 4 3 4 5)

pp-remove clj

(pp-remove n p)

(pp-remove n p coll)

Parametrized parallel remove, with max of n threads at once.

Accepts number of threads n, predicate p and a collection. If no collection is provided it produces a stateful transducer. Uses futures.

Why?

Free once pp-filter is defined.

How?

(vec (remove odd? [1 2 3])) => [2] (vec (pp-remove 1 odd? [1 2 3])) => [2]

ppfilter clj deprecated

Shim for pp-filter!

ppmap clj deprecated

Shim for pp-map!

ppmapcat clj deprecated

Shim for pp-mapcat!

roll-in clj

(roll-in seq-of-seqs)

(roll-in agg seq-of-seqs)

Takes sequence of sequences, returns map of maps: last item is value, butlast items are the key.

If agg callable is given -- it's used with update-in, otherwise entries action is assoc-in.

Why?

I just like maps.

How?

(roll-in [[:a :b 3] [:a :c 4] [:x :z 0]]) => {:a {:b 3 :c 4} :x {:z 0}} (roll-in (fnil + 0) [[:a :b 3] [:a :c 4] [:x :z 0] [:a :c 2]]) => {:a {:b 3 :c 6} :x {:z 0}}

roll-out clj

(roll-out map-of-maps)

(roll-out stop-at? map-of-maps)

Takes a map of maps, returns sequence of vectors.

The stop-at? predicate halts expansion on keys it returns true for.

Why?

I like maps, but vectors are also pretty neat.

How?

(set (roll-out {:a {:b 3 :c 4} :x {:z 0}})) ;; because ordering. => #{[:a :b 3] [:a :c 4] [:x :z 0]} (set (roll-out #(contains? % :b) {:a {:b 3 :c 4} :x {:z 0}})) => #{[:a {:b 3 :c 4}] [:x :z 0]}

row-adjuster clj

(row-adjuster default-row)

Creates a function that will return vectors of same length as default-row. Missing columns will be filled by defaults. Extra columns dropped.

Why?

Because *SV does not have to be well formed, numbers of column may vary.

How?

((row-adjuster [1 2 3]) [:x]) => [:x 2 3] ((row-adjuster [1 2 3]) [:1 :2 :3 :4]) => [:1 :2 :3]

row-field-unpacker clj

(row-field-unpacker delimiter a-row)

Takes a delimiter and a row, runs field unpacker over the row.

Why?

Partial it away and apply to nested CSV/TSV stuff.

How?

(row-field-unpacker , ["xnay" "ala,ma,kota" "unpackey") => ["xnay" ["ala" "ma" "kota"] "unpackey"]

vec->map clj

(vec->map header values)

(vec->map defaults header values)

Takes header and values vector, zipmaps, adds metadata. Allows defaults.

Why?

Singular operation for vecs->maps, with inputs attached as meta.

How? (vec->map [:k1 :k2] [1 2]) => {:k1 1, :k2 2} (vec->map {:kx 100} [:k1 :k2] [1 2]) => {:kx 100, :k1 1, :k2 2} (meta (vec->map {:kx 100} [:k1 :k2] [1 2])) => {:defaults {:kx 100}, :header [:k1 :k2], :values [1 2]}

vecs->maps clj

(vecs->maps a-seq)

(vecs->maps head tail)

(vecs->maps defaults head tail)

Takes header vector and tail, returns sequence of maps.

Zipmaps head with tail. Merges with defaults, doing nil substitution.

Why?

Maps are friendlier then vectors.

How?

(vecs->maps [[:k1 :k2] [1 2] [3 4]]) => ({:k1 1, :k2 2}, {:k1 3 :k2 4}) (vecs->maps [:k1 :k2] [[1 2] [3 4]]) => ({:k1 1, :k2 2}, {:k1 3 :k2 4}) (let [s [[:a :b] [1 2] [3 4]]] (vecs->maps (first s) (rest s))) => ({:a 1, :b 2}, {:a 3, :b 4}) (let [d {:x 100, :b 9} s [[:a :b] [1 nil] [3 4]]] (vecs->maps d (first s) (rest s))) => ({:a 1 :b 9 :x 100} {:a 3 :b 4 :x 100})

vecs-maker clj

(vecs-maker)

(vecs-maker defaults)

(vecs-maker defaults header-prep row-prep)

Accepts defaults and two output producing functions of header and row.

By default header-prep is vector of SCREAMING_SNAKE_CASE_STRING and row-prep is a vector of String with nil replaced by "NULL".

Bespoke header based on first record if sequence, so they better be uniform!

Why?

Quick and dirty dumps for spreadsheet consumption!

How? ((vecs-maker) [[:x :y :z] [{:x 0 :y 0} {:x 10 :y 10}]]) => (["X" "Y" "Z"] ["0" "0" "NULL"] ["10" "10" "NULL"]) ((vecs-maker {:z 0}) [[:x :y :z] [{:x 0 :y 0} {:x 10 :y 10}]]) => (["X" "Y" "Z"] ["0" "0" "0"] ["10" "10" "0"]) ((vecs-maker {:z 0} identity identity) [[:x :y :z] [{:x 0 :y 0} {:x 10 :y 10}]]) => ([:x :y :z] [0 0 0] [10 10 0]) ((vecs-maker nil identity identity) [[:x :y :z] [{:x 0 :y 0} {:x 10 :y 10}]]) => ([:x :y :z] [0 0 nil] [10 10 nil])