Liking cljdoc? Tell your friends :D
Clojure only.

scicloj.ml.dataset

This namespace contains functions which operate on a dataset and mostly return a dataset.

The namespaces scicloj.ml.metamorph and scicloj.ml.dataset contain functions with the same name. But they operate on either a context map (ns metamorph) or on a dataset (ns dataset)

The functions in tis namespace are re-exported from:

  • tabecloth.api
  • tech.v3.dataset.modelling
  • tech.v3.dataset.column-filters
This namespace contains functions which operate on a dataset
and mostly return a dataset.

The namespaces scicloj.ml.metamorph and scicloj.ml.dataset contain
functions with the same name. But they operate on either a context
map (ns metamorph) or on a dataset (ns dataset)

The functions in tis namespace are re-exported from:

* tabecloth.api
* tech.v3.dataset.modelling
* tech.v3.dataset.column-filters
raw docstring

->arrayclj

(->array ds colname)
(->array ds colname datatype)

Convert numerical column(s) to java array

Convert numerical column(s) to java array
raw docstring

add-columnclj

(add-column ds column-name column)
(add-column ds column-name column size-strategy)

Add or update (modify) column under column-name.

column can be sequence of values or generator function (which gets ds as input).

Add or update (modify) column under `column-name`.

`column` can be sequence of values or generator function (which gets `ds` as input).
raw docstring

add-columnsclj

(add-columns ds columns-map)
(add-columns ds columns-map size-strategy)

Add or updade (modify) columns defined in columns-map (mapping: name -> column)

Add or updade (modify) columns defined in `columns-map` (mapping: name -> column) 
raw docstring

add-or-replace-columnclj

(add-or-replace-column ds column-name column)
(add-or-replace-column ds column-name column size-strategy)

add-or-replace-columnsclj

(add-or-replace-columns ds columns-map)
(add-or-replace-columns ds columns-map size-strategy)

aggregateclj

(aggregate ds aggregator)
(aggregate ds
           aggregator
           {:keys [default-column-name-prefix ungroup? parallel?]
            :or {default-column-name-prefix "summary" ungroup? true}
            :as options})

Aggregate dataset by providing:

  • aggregation function
  • map with column names and functions
  • sequence of aggregation functions

Aggregation functions can return:

  • single value
  • seq of values
  • map of values with column names
Aggregate dataset by providing:

- aggregation function
- map with column names and functions
- sequence of aggregation functions

Aggregation functions can return:
- single value
- seq of values
- map of values with column names
raw docstring

aggregate-columnsclj

(aggregate-columns ds columns-selector column-aggregators)
(aggregate-columns ds columns-selector column-aggregators options)

Aggregates each column separately

Aggregates each column separately
raw docstring

anti-joinclj

(anti-join ds-left ds-right columns-selector)
(anti-join ds-left ds-right columns-selector options)

appendclj

(append ds & datasets)

as-regular-datasetclj

(as-regular-dataset ds)

Remove grouping tag

Remove grouping tag
raw docstring

asof-joinclj

(asof-join ds-left ds-right colname)
(asof-join ds-left ds-right colname options)

bindclj

(bind ds & datasets)

booleanclj

(boolean dataset)

Return a dataset containing only the boolean columns.

Return a dataset containing only the boolean columns.
raw docstring

by-rankclj

(by-rank ds columns-selector rank-predicate)
(by-rank ds
         columns-selector
         rank-predicate
         {:keys [desc? ties] :or {desc? true ties :dense}})

Select rows using rank on a column, ties are resolved using :dense method.

See R docs. Rank uses 0 based indexing.

Possible :ties strategies: :average, :first, :last, :random, :min, :max, :dense. :dense is the same as in data.table::frank from R

:desc? set to true (default) order descending before calculating rank

Select rows using `rank` on a column, ties are resolved using `:dense` method.

See [R docs](https://www.rdocumentation.org/packages/base/versions/3.6.1/topics/rank).
Rank uses 0 based indexing.

Possible `:ties` strategies: `:average`, `:first`, `:last`, `:random`, `:min`, `:max`, `:dense`.
`:dense` is the same as in `data.table::frank` from R

`:desc?` set to true (default) order descending before calculating rank
raw docstring

categoricalclj

(categorical dataset)

Return a dataset containing only the categorical columns.

Return a dataset containing only the categorical columns.
raw docstring

categorical->numberclj

(categorical->number dataset filter-fn-or-ds)
(categorical->number dataset filter-fn-or-ds table-args)
(categorical->number dataset filter-fn-or-ds table-args result-datatype)

Convert columns into a discrete , numeric representation See tech.v3.dataset.categorical/fit-categorical-map.

Convert columns into a discrete , numeric representation
See tech.v3.dataset.categorical/fit-categorical-map.
raw docstring

categorical->one-hotclj

(categorical->one-hot dataset filter-fn-or-ds)
(categorical->one-hot dataset filter-fn-or-ds table-args)
(categorical->one-hot dataset filter-fn-or-ds table-args result-datatype)

Convert string columns to numeric columns. See tech.v3.dataset.categorical/fit-one-hot

Convert string columns to numeric columns.
See tech.v3.dataset.categorical/fit-one-hot
raw docstring

CategoricalMapclj


cloneclj

(clone item)

Clone an object. Can clone anything convertible to a reader.

Clone an object.  Can clone anything convertible to a reader.
raw docstring

columnclj

(column dataset colname)

column-countclj

(column-count dataset)

column-filterclj

(column-filter dataset filter-fn)

Return a dataset with only the columns for which the filter function returns a truthy value.

Return a dataset with only the columns for which the filter function returns a truthy
value.
raw docstring

column-namesclj

(column-names ds)
(column-names ds columns-selector)
(column-names ds columns-selector meta-field)

column-values->categoricalclj

(column-values->categorical dataset src-column)

Given a column encoded via either string->number or one-hot, reverse map to the a sequence of the original string column values. In the case of one-hot mappings, src-column must be the original column name before the one-hot map

Given a column encoded via either string->number or one-hot, reverse
map to the a sequence of the original string column values.
In the case of one-hot mappings, src-column must be the original
column name before the one-hot map
raw docstring

columnsclj

(columns ds)
(columns ds result-type)

concatclj

(concat dataset & datasets)

concat-copyingclj

(concat-copying dataset & datasets)

convert-typesclj

(convert-types ds coltype-map-or-columns-selector)
(convert-types ds columns-selector new-types)

Convert type of the column to the other type.

Convert type of the column to the other type.
raw docstring

create-categorical-mapclj

(create-categorical-map lookup-table src-colname result-datatype)

datasetclj

(dataset)
(dataset data)
(dataset data
         {:keys [single-value-column-name column-names layout]
          :or {single-value-column-name :$value layout :as-columns}
          :as options})

Create dataset.

Dataset can be created from:

  • single value
  • map of values and/or sequences
  • sequence of maps
  • sequence of columns
  • file or url
Create `dataset`.

Dataset can be created from:

* single value
* map of values and/or sequences
* sequence of maps
* sequence of columns
* file or url
raw docstring

dataset->categorical-mapsclj

(dataset->categorical-maps dataset)

Given a dataset, return a map of column names to categorical label maps. This aids in inverting all of the label maps in a dataset. The source column name is src-column.

Given a dataset, return a map of column names to categorical label maps.
This aids in inverting all of the label maps in a dataset.
The source column name is src-column.
raw docstring

dataset->categorical-xformsclj

(dataset->categorical-xforms ds)

Given a dataset, return a map of column-name->xform information.

Given a dataset, return a map of column-name->xform information.
raw docstring

dataset->one-hot-mapsclj

(dataset->one-hot-maps dataset)

Given a dataset, return a sequence of applied on-hot transformations.

Given a dataset, return a sequence of applied on-hot transformations.
raw docstring

dataset->strclj

(dataset->str ds)
(dataset->str ds options)

Convert a dataset to a string. Prints a single line header and then calls dataset-data->str.

For options documentation see dataset-data->str.

Convert a dataset to a string.  Prints a single line header and then calls
dataset-data->str.

For options documentation see dataset-data->str.
raw docstring

dataset-nameclj

(dataset-name dataset)

dataset?clj

(dataset? ds)

Is ds a dataset type?

Is `ds` a `dataset` type?
raw docstring

datetimeclj

(datetime dataset)

Return a dataset containing only the datetime columns.

Return a dataset containing only the datetime columns.
raw docstring

differenceclj

(difference ds-left ds-right)
(difference ds-left ds-right options)

dropclj

(drop ds columns-selector rows-selector)

Drop columns and rows.

Drop columns and rows.
raw docstring

drop-columnsclj

(drop-columns ds)
(drop-columns ds columns-selector)
(drop-columns ds columns-selector meta-field)

Drop columns by (returns dataset):

  • name
  • sequence of names
  • map of names with new names (rename)
  • function which filter names (via column metadata)
Drop columns by (returns dataset):

- name
- sequence of names
- map of names with new names (rename)
- function which filter names (via column metadata)
raw docstring

drop-missingclj

(drop-missing ds)
(drop-missing ds columns-selector)

Drop rows with missing values

columns-selector selects columns to look at missing values

Drop rows with missing values

`columns-selector` selects columns to look at missing values
raw docstring

drop-rowsclj

(drop-rows ds)
(drop-rows ds rows-selector)
(drop-rows ds rows-selector {:keys [select-keys pre result-type parallel?]})

Drop rows using:

  • row id
  • seq of row ids
  • seq of true/false
  • fn with predicate
Drop rows using:

- row id
- seq of row ids
- seq of true/false
- fn with predicate
raw docstring

empty-ds?clj

(empty-ds? ds)

featureclj

(feature dataset)

Return a dataset container only the columns which have not been marked as inference columns.

Return a dataset container only the columns which have not been marked as inference
columns.
raw docstring

feature-ecountclj

(feature-ecount dataset)

Number of feature columns. Feature columns are columns that are not inference targets.

Number of feature columns.  Feature columns are columns that are not
inference targets.
raw docstring

fill-range-replaceclj

(fill-range-replace ds colname max-span)
(fill-range-replace ds colname max-span missing-strategy)
(fill-range-replace ds colname max-span missing-strategy missing-value)

firstclj

(first ds)

fit-categorical-mapclj

(fit-categorical-map dataset colname & [table-args res-dtype])

Given a column, map it into an numeric space via a discrete map of values to integers. This fits the categorical transformation onto the column and returns the transformation.

Given a column, map it into an numeric space via a discrete map of values
to integers.  This fits the categorical transformation onto the column and returns
the transformation.
raw docstring

fit-one-hotclj

(fit-one-hot dataset colname & [table-args res-dtype])

Fit a one hot transformation to a column. Returns a reusable transformation. Maps each unique value to a column with 1 every time the value appears in the original column and 0 otherwise.

Fit a one hot transformation to a column.  Returns a reusable transformation.
Maps each unique value to a column with 1 every time the value appears in the
original column and 0 otherwise.
raw docstring

fold-byclj

(fold-by ds columns-selector)
(fold-by ds columns-selector folding-function)

full-joinclj

(full-join ds-left ds-right columns-selector)
(full-join ds-left ds-right columns-selector options)

group-byclj

(group-by ds grouping-selector)
(group-by ds
          grouping-selector
          {:keys [select-keys result-type]
           :or {result-type :as-dataset select-keys :all}
           :as options})

Group dataset by:

  • column name
  • list of columns
  • map of keys and row indexes
  • function getting map of values

Options are:

  • select-keys - when grouping is done by function, you can limit fields to a select-keys seq.
  • result-type - return results as dataset (:as-dataset, default) or as map of datasets (:as-map) or as map of row indexes (:as-indexes) or as sequence of (sub)datasets
  • other parameters which are passed to dataset fn

When dataset is returned, meta contains :grouped? set to true. Columns in dataset:

  • name - group name
  • group-id - id of the group (int)
  • data - group as dataset
Group dataset by:

- column name
- list of columns
- map of keys and row indexes
- function getting map of values

Options are:

- select-keys - when grouping is done by function, you can limit fields to a `select-keys` seq.
- result-type - return results as dataset (`:as-dataset`, default) or as map of datasets (`:as-map`) or as map of row indexes (`:as-indexes`) or as sequence of (sub)datasets
- other parameters which are passed to `dataset` fn

When dataset is returned, meta contains `:grouped?` set to true. Columns in dataset:

- name - group name
- group-id - id of the group (int)
- data - group as dataset
raw docstring

grouped?clj

(grouped? ds)

Is dataset represents grouped dataset (result of group-by)?

Is `dataset` represents grouped dataset (result of `group-by`)?
raw docstring

groups->mapclj

(groups->map ds)

Convert grouped dataset to the map of groups

Convert grouped dataset to the map of groups
raw docstring

groups->seqclj

(groups->seq ds)

has-column?clj

(has-column? dataset column-name)

(head ds)
(head ds n)

inference-column?clj

(inference-column? col)

inference-target-column-namesclj

(inference-target-column-names ds)

Return the names of the columns that are inference targets.

Return the names of the columns that are inference targets.
raw docstring

inference-target-dsclj

(inference-target-ds dataset)

Given a dataset return reverse-mapped inference target columns or nil in the case where there are no inference targets.

Given a dataset return reverse-mapped inference target columns or nil
in the case where there are no inference targets.
raw docstring

inference-target-label-inverse-mapclj

(inference-target-label-inverse-map dataset & [label-columns])

Given options generated during ETL operations and annotated with :label-columns sequence container 1 label column, generate a reverse map that maps from a dataset value back to the label that generated that value.

Given options generated during ETL operations and annotated with :label-columns
sequence container 1 label column, generate a reverse map that maps from a dataset
value back to the label that generated that value.
raw docstring

inference-target-label-mapclj

(inference-target-label-map dataset & [label-columns])

infoclj

(info ds)
(info ds result-type)

inner-joinclj

(inner-join ds-left ds-right columns-selector)
(inner-join ds-left ds-right columns-selector options)

intersectclj

(intersect ds-left ds-right)
(intersect ds-left ds-right options)

intersectionclj

(intersection lhs-ds rhs-ds)

Return only columns for rhs for which an equivalently named column exists in lhs.

Return only columns for rhs for which an equivalently named column exists in lhs.
raw docstring

invert-categorical-mapclj

(invert-categorical-map dataset {:keys [src-column lookup-table]})

Invert a categorical map returning the column to the original set of values.

Invert a categorical map returning the column to the original set of values.
raw docstring

invert-one-hot-mapclj

(invert-one-hot-map dataset {:keys [one-hot-table src-column]})

Invert a one-hot transformation removing the one-hot columns and adding back the original column.

Invert a one-hot transformation removing the one-hot columns and adding back the
original column.
raw docstring

join-columnsclj

(join-columns ds target-column columns-selector)
(join-columns ds
              target-column
              columns-selector
              {:keys [separator missing-subst drop-columns? result-type
                      parallel?]
               :or {separator "-" drop-columns? true result-type :string}})

k-fold-datasetsclj

(k-fold-datasets dataset k)
(k-fold-datasets dataset
                 k
                 {:keys [randomize-dataset?] :or {randomize-dataset? true}})

Given 1 dataset, prepary K datasets using the k-fold algorithm. Randomize dataset defaults to true which will realize the entire dataset so use with care if you have large datasets.

Given 1 dataset, prepary K datasets using the k-fold algorithm.
Randomize dataset defaults to true which will realize the entire dataset
so use with care if you have large datasets.
raw docstring

labelsclj

(labels dataset)

Return the labels. The labels sequence is the reverse mapped inference column. This returns a single column of data or errors out.

Return the labels.  The labels sequence is the reverse mapped inference
column.  This returns a single column of data or errors out.
raw docstring

lastclj

(last ds)

left-joinclj

(left-join ds-left ds-right columns-selector)
(left-join ds-left ds-right columns-selector options)

map-columnsclj

(map-columns ds column-name map-fn)
(map-columns ds column-name columns-selector map-fn)
(map-columns ds column-name new-type columns-selector map-fn)

mark-as-groupclj

(mark-as-group ds)

Add grouping tag

Add grouping tag
raw docstring

metadata-filterclj

(metadata-filter dataset filter-fn)

Return a dataset with only the columns for which, given the column metadata, the filter function returns a truthy value.

Return a dataset with only the columns for which, given the column metadata,
the filter function returns a truthy value.
raw docstring

missingclj

(missing dataset)

Return a dataset with only columns have have missing values

Return a dataset with only columns have have missing values
raw docstring

model-typeclj

(model-type dataset & [column-name-seq])

Check the label column after dataset processing. Return either :regression :classification

Check the label column after dataset processing.
Return either
:regression
:classification
raw docstring

no-missingclj

(no-missing dataset)

Return a dataset with only columns that have no missing values.

Return a dataset with only columns that have no missing values.
raw docstring

num-inference-classesclj

(num-inference-classes dataset)

Given a dataset and correctly built options from pipeline operations, return the number of classes used for the label. Error if not classification dataset.

Given a dataset and correctly built options from pipeline operations,
return the number of classes used for the label.  Error if not classification
dataset.
raw docstring

numericclj

(numeric dataset)

Return a dataset containing only the numeric columns.

Return a dataset containing only the numeric columns.
raw docstring

of-datatypeclj

(of-datatype dataset datatype)

Return a dataset containing only the columns of a specific datatype.

Return a dataset containing only the columns of a specific datatype.
raw docstring

OneHotMapclj


order-byclj

(order-by ds columns-or-fn)
(order-by ds columns-or-fn comparators)
(order-by ds columns-or-fn comparators {:keys [parallel?]})

Order dataset by:

  • column name
  • columns (as sequence of names)
  • key-fn
  • sequence of columns / key-fn Additionally you can ask the order by:
  • :asc
  • :desc
  • custom comparator function
Order dataset by:
- column name
- columns (as sequence of names)
- key-fn
- sequence of columns / key-fn
Additionally you can ask the order by:
- :asc
- :desc
- custom comparator function
raw docstring

pivot->longerclj

(pivot->longer ds)
(pivot->longer ds columns-selector)
(pivot->longer
  ds
  columns-selector
  {:keys [target-columns value-column-name splitter drop-missing? datatypes]
   :or {target-columns :$column value-column-name :$value drop-missing? true}})

tidyr pivot_longer api

`tidyr` pivot_longer api
raw docstring

pivot->widerclj

(pivot->wider ds columns-selector value-columns)
(pivot->wider
  ds
  columns-selector
  value-columns
  {:keys [fold-fn concat-columns-with concat-value-with drop-missing?]
   :or {concat-columns-with "_" concat-value-with "-" drop-missing? true}})

predictionclj

(prediction dataset)

Return the columns of the dataset marked as predictions.

Return the columns of the dataset marked as predictions.
raw docstring

(print-dataset ds)
(print-dataset ds options)

probability-distributionclj

(probability-distribution dataset)

Return the columns of the dataset that comprise the probability distribution after classification.

Return the columns of the dataset that comprise the probability distribution
after classification.
raw docstring

probability-distributions->label-columnclj

(probability-distributions->label-column prob-ds dst-colname)

Given a dataset that has columns in which the column names describe labels and the rows describe a probability distribution, create a label column by taking the max value in each row and assign column that row value.

Given a dataset that has columns in which the column names describe labels and the
rows describe a probability distribution, create a label column by taking the max
value in each row and assign column that row value.
raw docstring

process-group-dataclj

(process-group-data ds f)
(process-group-data ds f parallel?)

rand-nthclj

(rand-nth ds)
(rand-nth ds {:keys [seed]})

randomclj

(random ds)
(random ds n)
(random ds n {:keys [repeat? seed] :or {repeat? true}})

read-nippyclj

(read-nippy filename)

rename-columnsclj

(rename-columns ds columns-mapping)
(rename-columns ds columns-selector columns-map-fn)

Rename columns with provided old -> new name map

Rename columns with provided old -> new name map
raw docstring

reorder-columnsclj

(reorder-columns ds columns-selector & columns-selectors)

Reorder columns using column selector(s). When column names are incomplete, the missing will be attached at the end.

Reorder columns using column selector(s). When column names are incomplete, the missing will be attached at the end.
raw docstring

replace-missingclj

(replace-missing ds)
(replace-missing ds strategy)
(replace-missing ds columns-selector strategy)
(replace-missing ds columns-selector strategy value)

reverse-map-categorical-xformsclj

(reverse-map-categorical-xforms dataset)

Given a dataset where we have converted columns from a categorical representation to either a numeric reprsentation or a one-hot representation, reverse map back to the original dataset given the reverse mapping of label->number in the column's metadata.

Given a dataset where we have converted columns from a categorical representation
to either a numeric reprsentation or a one-hot representation, reverse map
back to the original dataset given the reverse mapping of label->number in
the column's metadata.
raw docstring

right-joinclj

(right-join ds-left ds-right columns-selector)
(right-join ds-left ds-right columns-selector options)

row-countclj

(row-count dataset-or-col)

rowsclj

(rows ds)
(rows ds result-type)

selectclj

(select ds columns-selector rows-selector)

Select columns and rows.

Select columns and rows.
raw docstring

select-columnsclj

(select-columns ds)
(select-columns ds columns-selector)
(select-columns ds columns-selector meta-field)

Select columns by (returns dataset):

  • name
  • sequence of names
  • map of names with new names (rename)
  • function which filter names (via column metadata)
Select columns by (returns dataset):

- name
- sequence of names
- map of names with new names (rename)
- function which filter names (via column metadata)
raw docstring

select-missingclj

(select-missing ds)
(select-missing ds columns-selector)

Select rows with missing values

columns-selector selects columns to look at missing values

Select rows with missing values

`columns-selector` selects columns to look at missing values
raw docstring

select-rowsclj

(select-rows ds)
(select-rows ds rows-selector)
(select-rows ds rows-selector {:keys [select-keys pre result-type parallel?]})

Select rows using:

  • row id
  • seq of row ids
  • seq of true/false
  • fn with predicate
Select rows using:

- row id
- seq of row ids
- seq of true/false
- fn with predicate
raw docstring

semi-joinclj

(semi-join ds-left ds-right columns-selector)
(semi-join ds-left ds-right columns-selector options)

separate-columnclj

(separate-column ds column separator)
(separate-column ds column target-columns separator)
(separate-column ds
                 column
                 target-columns
                 separator
                 {:keys [missing-subst drop-column? parallel?]
                  :or {missing-subst ""}})

set-dataset-nameclj

(set-dataset-name dataset ds-name)

set-inference-targetclj

(set-inference-target dataset target-name-or-target-name-seq)

Set the inference target on the column. This sets the :column-type member of the column metadata to :inference-target?.

Set the inference target on the column.  This sets the :column-type member
of the column metadata to :inference-target?.
raw docstring

shapeclj

(shape ds)

Returns shape of the dataset [rows, cols]

Returns shape of the dataset [rows, cols]
raw docstring

shuffleclj

(shuffle ds)
(shuffle ds {:keys [seed]})

splitclj

(split ds)
(split ds split-type)
(split ds
       split-type
       {:keys [seed parallel? shuffle?] :or {shuffle? true} :as opts})

Split given dataset into 2 or more (holdout) splits

As the result two new columns are added:

  • :$split-name - with subgroup name
  • :$split-id - fold id/repetition id

split-type can be one of the following:

  • :kfold - k-fold strategy, :k defines number of folds (defaults to 5), produces k splits
  • :bootstrap - :ratio defines ratio of observations put into result (defaults to 1.0), produces 1 split
  • :holdout - split into two parts with given ratio (defaults to 2/3), produces 1 split
  • :loo - leave one out, produces the same number of splits as number of observations

:holdout can accept also probabilites or ratios and can split to more than 2 subdatasets

Additionally you can provide:

  • :seed - for random number generator
  • :repeats - repeat procedure :repeats times
  • :partition-selector - same as in group-by for stratified splitting to reflect dataset structure in splits.
  • :split-names names of subdatasets different than default, ie. [:train :test :split-2 ...]
  • :split-col-name - a column where name of split is stored, either :train or :test values (default: :$split-name)
  • :split-id-col-name - a column where id of the train/test pair is stored (default: :$split-id)

Rows are shuffled before splitting.

In case of grouped dataset each group is processed separately.

See more

Split given dataset into 2 or more (holdout) splits

As the result two new columns are added:

* `:$split-name` - with subgroup name
* `:$split-id` - fold id/repetition id

`split-type` can be one of the following:

* `:kfold` - k-fold strategy, `:k` defines number of folds (defaults to `5`), produces `k` splits
* `:bootstrap` - `:ratio` defines ratio of observations put into result (defaults to `1.0`), produces `1` split
* `:holdout` - split into two parts with given ratio (defaults to `2/3`), produces `1` split
* `:loo` - leave one out, produces the same number of splits as number of observations

`:holdout` can accept also probabilites or ratios and can split to more than 2 subdatasets

Additionally you can provide:

* `:seed` - for random number generator
* `:repeats` - repeat procedure `:repeats` times
* `:partition-selector` - same as in `group-by` for stratified splitting to reflect dataset structure in splits.
* `:split-names` names of subdatasets different than default, ie. `[:train :test :split-2 ...]`
* `:split-col-name` - a column where name of split is stored, either `:train` or `:test` values (default: `:$split-name`)
* `:split-id-col-name` - a column where id of the train/test pair is stored (default: `:$split-id`)

Rows are shuffled before splitting.

In case of grouped dataset each group is processed separately.

See [more](https://www.mitpressjournals.org/doi/pdf/10.1162/EVCO_a_00069)
raw docstring

split->seqclj

(split->seq ds)
(split->seq ds split-type)
(split->seq ds
            split-type
            {:keys [split-col-name split-id-col-name]
             :or {split-col-name :$split-name split-id-col-name :$split-id}
             :as opts})

Returns split as a sequence of train/test datasets or map of sequences (grouped dataset)

Returns split as a sequence of train/test datasets or map of sequences (grouped dataset)
raw docstring

stringclj

(string dataset)

Return a dataset containing only the string columns.

Return a dataset containing only the string columns.
raw docstring

tailclj

(tail ds)
(tail ds n)

targetclj

(target dataset)

Return a dataset containing only the columns that have been marked as inference targets.

Return a dataset containing only the columns that have been marked as inference
targets.
raw docstring

train-test-splitclj

(train-test-split dataset)
(train-test-split dataset
                  {:keys [randomize-dataset? train-fraction]
                   :or {randomize-dataset? true train-fraction 0.7}})

Probabilistically split the dataset returning a map of {:train-ds :test-ds}.

Probabilistically split the dataset returning a map of `{:train-ds :test-ds}`.
raw docstring

transform-categorical-mapclj

(transform-categorical-map dataset fit-data)

Apply a categorical mapping transformation fit with fit-categorical-map.

Apply a categorical mapping transformation fit with fit-categorical-map.
raw docstring

transform-one-hotclj

(transform-one-hot dataset one-hot-fit-data)

Apply a one-hot transformation to a dataset

Apply a one-hot transformation to a dataset
raw docstring

ungroupclj

(ungroup ds)
(ungroup ds
         {:keys [order? add-group-as-column add-group-id-as-column separate?
                 dataset-name parallel?]
          :or {separate? true}})

Concat groups into dataset.

When add-group-as-column or add-group-id-as-column is set to true or name(s), columns with group name(s) or group id is added to the result.

Before joining the groups groups can be sorted by group name.

Concat groups into dataset.

When `add-group-as-column` or `add-group-id-as-column` is set to `true` or name(s), columns with group name(s) or group id is added to the result.

Before joining the groups groups can be sorted by group name.
raw docstring

unionclj

(union ds & datasets)

unique-byclj

(unique-by ds)
(unique-by ds columns-selector)
(unique-by
  ds
  columns-selector
  {:keys [strategy select-keys parallel?] :or {strategy :first} :as options})

unmark-groupclj

(unmark-group ds)

Remove grouping tag

Remove grouping tag
raw docstring

unrollclj

(unroll ds columns-selector)
(unroll ds columns-selector options)

update-columnsclj

(update-columns ds columns-map)
(update-columns ds columns-selector update-functions)

write!clj

(write! dataset output-path)
(write! dataset output-path options)

Write a dataset out to a file. Supported forms are:

(ds/write! test-ds "test.csv")
(ds/write! test-ds "test.tsv")
(ds/write! test-ds "test.tsv.gz")
(ds/write! test-ds "test.nippy")
(ds/write! test-ds out-stream)

Options:

  • :max-chars-per-column - csv,tsv specific, defaults to 65536 - values longer than this will cause an exception during serialization.
  • :max-num-columns - csv,tsv specific, defaults to 8192 - If the dataset has more than this number of columns an exception will be thrown during serialization.
  • :quoted-columns - csv specific - sequence of columns names that you would like to always have quoted.
  • :file-type - Manually specify the file type. This is usually inferred from the filename but if you pass in an output stream then you will need to specify the file type.
  • :headers? - if csv headers are written, defaults to true.
Write a dataset out to a file.  Supported forms are:

```clojure
(ds/write! test-ds "test.csv")
(ds/write! test-ds "test.tsv")
(ds/write! test-ds "test.tsv.gz")
(ds/write! test-ds "test.nippy")
(ds/write! test-ds out-stream)
```

Options:

  * `:max-chars-per-column` - csv,tsv specific, defaults to 65536 - values longer than this will
     cause an exception during serialization.
  * `:max-num-columns` - csv,tsv specific, defaults to 8192 - If the dataset has more than this number of
     columns an exception will be thrown during serialization.
  * `:quoted-columns` - csv specific - sequence of columns names that you would like to always have quoted.
  * `:file-type` - Manually specify the file type.  This is usually inferred from the filename but if you
     pass in an output stream then you will need to specify the file type.
  * `:headers?` - if csv headers are written, defaults to true.
raw docstring

write-csv!clj


write-nippy!clj

(write-nippy! ds filename)

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close