clj-djl.dataframe — clj-djl/dataframe 0.1.2

->dataframe clj

(->dataframe dataframe)

(->dataframe dataframe {:keys [table-name dataset-name] :as options})

->ndarray clj

(->ndarray ndm dataframe)

Convert dataframe to NDArray

add-column clj

(add-column dataset column)

Add a new column. Error if name collision

add-or-update-column clj

(add-or-update-column dataset column)

(add-or-update-column dataset colname column)

If column exists, replace. Else append new column.

assoc-ds clj

(assoc-ds dataset cname cdata & args)

If dataset is not nil, calls clojure.core/assoc. Else creates a new empty dataset and then calls clojure.core/assoc. Guaranteed to return a dataset (unlike assoc).

brief clj

(brief ds)

(brief ds options)

Get a brief description, in mapseq form of a dataset. A brief description is the mapseq form of descriptive stats.

categorical->one-hot clj

(categorical->one-hot dataset filter-fn-or-ds)

(categorical->one-hot dataset filter-fn-or-ds table-args)

(categorical->one-hot dataset filter-fn-or-ds table-args result-datatype)

Convert string columns to numeric columns. See tech.v3.dataset.categorical/fit-one-hot

column clj

(column dataset colname)

column-count clj

(column-count dataset)

column-names clj

(column-names dataset)

In-order sequence of column names

columns clj

(columns dataset)

Return sequence of all columns in dataset.

columns-with-missing-seq clj

(columns-with-missing-seq dataset)

Return a sequence of:

  {:column-name column-name
   :missing-count missing-count
  }

or nil of no columns are missing data.

concat clj

(concat dataset & datasets)

Concatenate datasets in place. See also concat-copying as it may be more efficient for your use case.

concat-copying clj

(concat-copying dataset & datasets)

Concatenate datasets into a new dataset copying data. Respects missing values. Datasets must all have the same columns. Result column datatypes will be a widening cast of the datatypes.

concat-inplace clj

(concat-inplace dataset & datasets)

Concatenate datasets in place. Respects missing values. Datasets must all have the same columns. Result column datatypes will be a widening cast of the datatypes.

data->dataframe clj

dataframe->data clj

dataframe-name clj

drop-columns clj

(drop-columns dataset col-name-seq)

Same as remove-columns

drop-missing clj

(drop-missing dataset-or-col)

Remove missing entries by simply selecting out the missing indexes

drop-rows clj

(drop-rows dataset-or-col row-indexes)

Drop rows from dataset or column

ensure-array-backed clj

(ensure-array-backed ds)

(ensure-array-backed ds {:keys [unpack?] :or {unpack? true}})

Ensure the column data in the dataset is stored in pure java arrays. This is sometimes necessary for interop with other libraries and this operation will force any lazy computations to complete. This also clears the missing set for each column and writes the missing values to the new arrays.

Columns that are already array backed and that have no missing values are not changed and retuned.

The postcondition is that dtype/->array will return a java array in the appropriate datatype for each column.

options - :unpack? - unpack packed datetime types. Defaults to true

filter clj

(filter dataset predicate)

dataset->dataset transformation. Predicate is passed a map of colname->column-value.

filter-column clj

(filter-column dataset colname predicate)

Filter a given column by a predicate. Predicate is passed column values. If predicate is not an instance of Ifn it is treated as a value and will be used as if the predicate is #(= value %). Returns a dataset.

group-by clj

(group-by dataset key-fn)

Produce a map of key-fn-value->dataset. key-fn is a function taking a map of colname->column-value. Selecting which columns are used in the key-fn using column-name-seq is optional but will greatly improve performance.

group-by->indexes clj

(group-by->indexes dataset key-fn)

(Non-lazy) - Group a dataset and return a map of key-fn-value->indexes where indexes is an in-order contiguous group of indexes.

group-by-column clj

(group-by-column dataset colname)

Return a map of column-value->dataset.

group-by-column->indexes clj

(group-by-column->indexes dataset colname)

(Non-lazy) - Group a dataset by a column return a map of column-val->indexes where indexes is an in-order contiguous group of indexes.

has-column? clj

(has-column? dataset column-name)

head clj

(head dataset)

(head dataset n)

Get the first n row of a dataset. Equivalent to `(select-rows ds (range n)). Arguments are reversed, however, so this can be used in ->> operators.

missing clj

(missing dataset-or-col)

Given a dataset or a column, return the missing set as a roaring bitmap

new-column clj

(new-column name data)

(new-column name data metadata)

(new-column name data metadata missing)

Create a new column. Data will scanned for missing values unless the full 4-argument pathway is used.

order-column-names clj

(order-column-names dataset colname-seq)

Order a sequence of columns names so they match the order in the original dataset. Missing columns are placed last.

remove-column clj

(remove-column dataset col-name)

Same as:

(dissoc dataset col-name)

remove-columns clj

(remove-columns dataset colname-seq)

Same as drop-columns

remove-rows clj

(remove-rows dataset-or-col row-indexes)

Same as drop-rows.

rename-columns clj

(rename-columns dataset colname-map)

Rename columns using a map. Does not reorder columns.

replace-missing clj

(replace-missing df)

(replace-missing df strategy)

(replace-missing df col-sel strategy)

Replace missing with:

• builtin strategys: :mid :up :down and :lerp
• value
• or column function with missing slot dropped

row-count clj

(row-count dataset-or-col)

select clj

(select dataset colname-seq index-seq)

Reorder/trim dataset according to this sequence of indexes. Returns a new dataset. colname-seq - one of:

• :all - all the columns
• sequence of column names - those columns in that order.
• implementation of java.util.Map - column order is dictate by map iteration order selected columns are subsequently named after the corresponding value in the map. similar to rename-columns except this trims the result to be only the columns in the map. index-seq - either keyword :all or list of indexes. May contain duplicates.

select-by-index clj

(select-by-index dataframe row-index col-index)

Select a sub-dataframe by seq of row index and column index

select-columns clj

(select-columns dataset col-name-seq)

Select columns from the dataset by seq of column names or :all.

select-columns-by-index clj

(select-columns-by-index dataset col-index)

Select columns from the dataset by seq of index(includes negative) or :all.

See documentation for select-by-index.

select-rows clj

(select-rows dataset-or-col row-indexes)

Select rows from the dataset or column.

select-rows-by-index clj

(select-rows-by-index dataset-or-col row-index)

Select rows from the dataset or column by seq of index(includes negative) or :all.

See documentation for select-by-index.

set-dataframe-name clj

shape clj

(shape dataframe)

Get the shape of dataframe, in row major way

sort-by clj

(sort-by dataset key-fn)

(sort-by dataset key-fn compare-fn)

Sort a dataset by a key-fn and compare-fn.

sort-by-column clj

(sort-by-column dataset colname)

(sort-by-column dataset colname compare-fn)

Sort a dataset by a given column using the given compare fn.

tail clj

(tail dataset)

(tail dataset n)

Get the last n rows of a dataset. Equivalent to `(select-rows ds (range ...)). Argument order is dataset-last, however, so this can be used in ->> operators.

take-nth clj

(take-nth dataset n-val)

unique-by clj

(unique-by dataset map-fn)

(unique-by dataset
           {:keys [keep-fn]
            :or {keep-fn (fn* [p1__25510# p2__25509#] (first p2__25509#))}
            :as _options}
           map-fn)

Map-fn function gets passed map for each row, rows are grouped by the return value. Keep-fn is used to decide the index to keep.

:keep-fn - Function from key,idx-seq->idx. Defaults to #(first %2).

unique-by-column clj

(unique-by-column dataset colname)

(unique-by-column dataset
                  {:keys [keep-fn]
                   :or {keep-fn (fn* [p1__25523# p2__25522#]
                                     (first p2__25522#))}
                   :as _options}
                  colname)

Map-fn function gets passed map for each row, rows are grouped by the return value. Keep-fn is used to decide the index to keep.

:keep-fn - Function from key, idx-seq->idx. Defaults to #(first %2).

unordered-select clj

(unordered-select dataset colname-seq index-seq)

Perform a selection but use the order of the columns in the existing table; do not reorder the columns based on colname-seq. Useful when doing selection based on sets or persistent hash maps.

update-column clj

(update-column dataset col-name update-fn)

Update a column returning a new dataset. update-fn is a column->column transformation. Error if column does not exist.

update-columns clj

(update-columns dataframe col-name-seq-or-fn update-fn)

Update a sequence of columns selected by column name seq or column selector function.