tablecloth.api — scicloj/tablecloth 6.031

->array clj

(->array ds colname)

(->array ds colname datatype)

Convert numerical column(s) to java array

add-column clj

(add-column ds column-name column)

(add-column ds column-name column size-strategy)

Add or update (modify) column under column-name.

column can be sequence of values or generator function (which gets ds as input).

• ds - a dataset
• column-name - if it's existing column name, column will be replaced
• column - can be column (from other dataset), sequence, single value or function. Too big columns are always trimmed. Too small are cycled or extended with missing values (according to size-strategy argument)
• size-strategy (optional) - when new column is shorter than dataset row count, following strategies are applied:
  • :cycle - repeat data
  • :na - append missing values
  • :strict - (default) throws an exception when sizes mismatch

add-columns clj

(add-columns ds columns-map)

(add-columns ds columns-map size-strategy)

Add or updade (modify) columns defined in columns-map (mapping: name -> column)

add-or-replace-column clj

(add-or-replace-column ds column-name column)

(add-or-replace-column ds column-name column size-strategy)

add-or-replace-columns clj

(add-or-replace-columns ds columns-map)

(add-or-replace-columns ds columns-map size-strategy)

aggregate clj

(aggregate ds aggregator)

(aggregate ds
           aggregator
           {:keys [default-column-name-prefix ungroup? parallel?]
            :or {default-column-name-prefix "summary" ungroup? true}
            :as options})

Aggregate dataset by providing:

• aggregation function
• map with column names and functions
• sequence of aggregation functions

Aggregation functions can return:

• single value
• seq of values
• map of values with column names

aggregate-columns clj

(aggregate-columns ds columns-selector column-aggregators)

(aggregate-columns ds columns-selector column-aggregators options)

Aggregates each column separately

anti-join clj

(anti-join ds-left ds-right columns-selector)

(anti-join ds-left ds-right columns-selector options)

append clj

(append ds & datasets)

as-regular-dataset clj

(as-regular-dataset ds)

Remove grouping tag

asof-join clj

(asof-join ds-left ds-right colname)

(asof-join ds-left ds-right colname options)

bind clj

(bind ds & datasets)

by-rank clj

(by-rank ds columns-selector rank-predicate)

(by-rank ds
         columns-selector
         rank-predicate
         {:keys [desc? ties] :or {desc? true ties :dense}})

Select rows using rank on a column, ties are resolved using :dense method.

See R docs. Rank uses 0 based indexing.

Possible :ties strategies: :average, :first, :last, :random, :min, :max, :dense. :dense is the same as in data.table::frank from R

:desc? set to true (default) order descending before calculating rank

clone clj

(clone item)

Clone an object. Can clone anything convertible to a reader.

column clj

(column dataset colname)

column-count clj

(column-count dataset)

column-names clj

(column-names ds)

(column-names ds columns-selector)

(column-names ds columns-selector meta-field)

columns clj

(columns ds)

(columns ds result-type)

Returns columns of dataset. Result type can be any of:

• :as-map
• :as-double-arrays
• :as-seqs

concat clj

(concat dataset & datasets)

concat-copying clj

(concat-copying dataset & datasets)

convert-types clj

(convert-types ds coltype-map-or-columns-selector)

(convert-types ds columns-selector new-types)

Convert type of the column to the other type.

dataset clj

(dataset)

(dataset data)

(dataset data
         {:keys [single-value-column-name column-names layout dataset-name]
          :or {single-value-column-name :$value layout :as-rows}
          :as options})

Create dataset.

Dataset can be created from:

• single value
• map of values and/or sequences
• sequence of maps
• sequence of columns
• file or url

dataset->str clj

(dataset->str ds)

(dataset->str ds options)

Convert a dataset to a string. Prints a single line header and then calls dataset-data->str.

For options documentation see dataset-data->str.

dataset-name clj

(dataset-name dataset)

dataset? clj

(dataset? ds)

Is ds a dataset type?

difference clj

(difference ds-left ds-right)

(difference ds-left ds-right options)

drop clj

(drop ds columns-selector rows-selector)

Drop columns and rows.

drop-columns clj

(drop-columns ds)

(drop-columns ds columns-selector)

(drop-columns ds columns-selector meta-field)

Drop columns by (returns dataset):

• name
• sequence of names
• map of names with new names (rename)
• function which filter names (via column metadata)

drop-missing clj

(drop-missing ds)

(drop-missing ds columns-selector)

Drop rows with missing values

columns-selector selects columns to look at missing values

drop-rows clj

(drop-rows ds)

(drop-rows ds rows-selector)

(drop-rows ds rows-selector {:keys [select-keys pre result-type parallel?]})

Drop rows using:

• row id
• seq of row ids
• seq of true/false
• fn with predicate

empty-ds? clj

(empty-ds? ds)

fill-range-replace clj

(fill-range-replace ds colname max-span)

(fill-range-replace ds colname max-span missing-strategy)

(fill-range-replace ds colname max-span missing-strategy missing-value)

first clj

(first ds)

fold-by clj

(fold-by ds columns-selector)

(fold-by ds columns-selector folding-function)

full-join clj

(full-join ds-left ds-right columns-selector)

(full-join ds-left ds-right columns-selector options)

group-by clj

(group-by ds grouping-selector)

(group-by ds
          grouping-selector
          {:keys [select-keys result-type]
           :or {result-type :as-dataset select-keys :all}
           :as options})

Group dataset by:

• column name
• list of columns
• map of keys and row indexes
• function getting map of values

Options are:

• select-keys - when grouping is done by function, you can limit fields to a select-keys seq.
• result-type - return results as dataset (:as-dataset, default) or as map of datasets (:as-map) or as map of row indexes (:as-indexes) or as sequence of (sub)datasets
• other parameters which are passed to dataset fn

When dataset is returned, meta contains :grouped? set to true. Columns in dataset:

• name - group name
• group-id - id of the group (int)
• data - group as dataset

grouped? clj

(grouped? ds)

Is dataset represents grouped dataset (result of group-by)?

groups->map clj

(groups->map ds)

Convert grouped dataset to the map of groups

groups->seq clj

(groups->seq ds)

has-column? clj

(has-column? dataset column-name)

head clj

(head ds)

(head ds n)

info clj

(info ds)

(info ds result-type)

inner-join clj

(inner-join ds-left ds-right columns-selector)

(inner-join ds-left ds-right columns-selector options)

intersect clj

(intersect ds-left ds-right)

(intersect ds-left ds-right options)

join-columns clj

(join-columns ds target-column columns-selector)

(join-columns ds
              target-column
              columns-selector
              {:keys [separator missing-subst drop-columns? result-type
                      parallel?]
               :or {separator "-" drop-columns? true result-type :string}})

last clj

(last ds)

left-join clj

(left-join ds-left ds-right columns-selector)

(left-join ds-left ds-right columns-selector options)

let-dataset clj macro

(let-dataset bindings)

(let-dataset bindings options)

map-columns clj

(map-columns ds column-name map-fn)

(map-columns ds column-name columns-selector map-fn)

(map-columns ds column-name new-type columns-selector map-fn)

mark-as-group clj

(mark-as-group ds)

Add grouping tag

order-by clj

(order-by ds columns-or-fn)

(order-by ds columns-or-fn comparators)

(order-by ds columns-or-fn comparators {:keys [parallel?]})

Order dataset by:

• column name
• columns (as sequence of names)
• key-fn
• sequence of columns / key-fn Additionally you can ask the order by:
• :asc
• :desc
• custom comparator function

pivot->longer clj

(pivot->longer ds)

(pivot->longer ds columns-selector)

(pivot->longer
  ds
  columns-selector
  {:keys [target-columns value-column-name splitter drop-missing? datatypes]
   :or {target-columns :$column value-column-name :$value drop-missing? true}})

tidyr pivot_longer api

pivot->wider clj

(pivot->wider ds columns-selector value-columns)

(pivot->wider
  ds
  columns-selector
  value-columns
  {:keys [fold-fn concat-columns-with concat-value-with drop-missing?]
   :or {concat-columns-with "_" concat-value-with "-" drop-missing? true}})

print-dataset clj

(print-dataset ds)

(print-dataset ds options)

process-group-data clj

(process-group-data ds f)

(process-group-data ds f parallel?)

rand-nth clj

(rand-nth ds)

(rand-nth ds {:keys [seed]})

random clj

(random ds)

(random ds n)

(random ds n {:keys [repeat? seed] :or {repeat? true}})

read-nippy clj

(read-nippy filename)

rename-columns clj

(rename-columns ds columns-mapping)

(rename-columns ds columns-selector columns-map-fn)

Rename columns with provided old -> new name map

reorder-columns clj

(reorder-columns ds columns-selector & columns-selectors)

Reorder columns using column selector(s). When column names are incomplete, the missing will be attached at the end.

replace-missing clj

(replace-missing ds)

(replace-missing ds strategy)

(replace-missing ds columns-selector strategy)

(replace-missing ds columns-selector strategy value)

right-join clj

(right-join ds-left ds-right columns-selector)

(right-join ds-left ds-right columns-selector options)

row-count clj

(row-count dataset-or-col)

rows clj

(rows ds)

(rows ds result-type)

Returns rows of dataset. Result type can be any of:

• :as-maps
• :as-double-arrays
• :as-seqs

select clj

(select ds columns-selector rows-selector)

Select columns and rows.

select-columns clj

(select-columns ds)

(select-columns ds columns-selector)

(select-columns ds columns-selector meta-field)

Select columns by (returns dataset):

• name
• sequence of names
• map of names with new names (rename)
• function which filter names (via column metadata)

select-missing clj

(select-missing ds)

(select-missing ds columns-selector)

Select rows with missing values

columns-selector selects columns to look at missing values

select-rows clj

(select-rows ds)

(select-rows ds rows-selector)

(select-rows ds rows-selector {:keys [select-keys pre result-type parallel?]})

Select rows using:

• row id
• seq of row ids
• seq of true/false
• fn with predicate

semi-join clj

(semi-join ds-left ds-right columns-selector)

(semi-join ds-left ds-right columns-selector options)

separate-column clj

(separate-column ds column separator)

(separate-column ds column target-columns separator)

(separate-column ds
                 column
                 target-columns
                 separator
                 {:keys [missing-subst drop-column? parallel?]
                  :or {missing-subst ""}})

set-dataset-name clj

(set-dataset-name dataset ds-name)

shape clj

(shape ds)

Returns shape of the dataset [rows, cols]

shuffle clj

(shuffle ds)

(shuffle ds {:keys [seed]})

split clj

(split ds)

(split ds split-type)

(split ds
       split-type
       {:keys [seed parallel? shuffle?] :or {shuffle? true} :as opts})

Split given dataset into 2 or more (holdout) splits

As the result two new columns are added:

• :$split-name - with subgroup name
• :$split-id - fold id/repetition id

split-type can be one of the following:

• :kfold - k-fold strategy, :k defines number of folds (defaults to 5), produces k splits
• :bootstrap - :ratio defines ratio of observations put into result (defaults to 1.0), produces 1 split
• :holdout - split into two parts with given ratio (defaults to 2/3), produces 1 split
• :loo - leave one out, produces the same number of splits as number of observations

:holdout can accept also probabilites or ratios and can split to more than 2 subdatasets

Additionally you can provide:

• :seed - for random number generator
• :repeats - repeat procedure :repeats times
• :partition-selector - same as in group-by for stratified splitting to reflect dataset structure in splits.
• :split-names names of subdatasets different than default, ie. [:train :test :split-2 ...]
• :split-col-name - a column where name of split is stored, either :train or :test values (default: :$split-name)
• :split-id-col-name - a column where id of the train/test pair is stored (default: :$split-id)

Rows are shuffled before splitting.

In case of grouped dataset each group is processed separately.

See more

split->seq clj

(split->seq ds)

(split->seq ds split-type)

(split->seq ds
            split-type
            {:keys [split-col-name split-id-col-name]
             :or {split-col-name :$split-name split-id-col-name :$split-id}
             :as opts})

Returns split as a sequence of train/test datasets or map of sequences (grouped dataset)

tail clj

(tail ds)

(tail ds n)

ungroup clj

(ungroup ds)

(ungroup ds
         {:keys [order? add-group-as-column add-group-id-as-column separate?
                 dataset-name parallel?]
          :or {separate? true}})

Concat groups into dataset.

When add-group-as-column or add-group-id-as-column is set to true or name(s), columns with group name(s) or group id is added to the result.

Before joining the groups groups can be sorted by group name.

union clj

(union ds & datasets)

unique-by clj

(unique-by ds)

(unique-by ds columns-selector)

(unique-by
  ds
  columns-selector
  {:keys [strategy select-keys parallel?] :or {strategy :first} :as options})

unmark-group clj

(unmark-group ds)

Remove grouping tag

unroll clj

(unroll ds columns-selector)

(unroll ds columns-selector options)

update-columns clj

(update-columns ds columns-map)

(update-columns ds columns-selector update-functions)

without-grouping-> clj macro

(without-grouping-> ds & r)

write! clj

(write! dataset output-path)

(write! dataset output-path options)

Write a dataset out to a file. Supported forms are:

(ds/write! test-ds "test.csv")
(ds/write! test-ds "test.tsv")
(ds/write! test-ds "test.tsv.gz")
(ds/write! test-ds "test.nippy")
(ds/write! test-ds out-stream)

Options:

• :max-chars-per-column - csv,tsv specific, defaults to 65536 - values longer than this will cause an exception during serialization.
• :max-num-columns - csv,tsv specific, defaults to 8192 - If the dataset has more than this number of columns an exception will be thrown during serialization.
• :quoted-columns - csv specific - sequence of columns names that you would like to always have quoted.
• :file-type - Manually specify the file type. This is usually inferred from the filename but if you pass in an output stream then you will need to specify the file type.
• :headers? - if csv headers are written, defaults to true.

write-csv! clj deprecated

write-nippy! clj

(write-nippy! ds filename)