scicloj.ml.smile.metamorph — scicloj/scicloj.ml.smile 5.06

bow->something-sparse clj

(bow->something-sparse bow-col indices-col bow->sparse-fn options)

Converts a bag-of-word column bow-col to a sparse data column indices-col. The exact transformation to the sparse representtaion is given by bow->sparse-fn

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	:scicloj.ml.smile.metamorph/bow->sparse-vocabulary

bow->sparse-array clj

(bow->sparse-array bow-col indices-col)

(bow->sparse-array bow-col indices-col options)

Converts a bag-of-word column bow-col to sparse indices column indices-col, as needed by the Maxent model. Options can be of:

create-vocab-fn A function which converts the bow map to a list of tokens. Defaults to scicloj.ml.smile.nlp/create-vocab-all

The sparse data is represented as primitive int arrays, of which entries are the indices against the vocabulary of the present tokens.

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	:scicloj.ml.smile.metamorph/bow->sparse-vocabulary

bow->SparseArray clj

(bow->SparseArray bow-col indices-col)

(bow->SparseArray bow-col indices-col options)

Converts a bag-of-word column bow-col to sparse indices column indices-col, as needed by the discrete naive bayes model.

Options can be of:

create-vocab-fn A function which converts the bow map to a list of tokens. Defaults to scicloj.ml.smile.nlp/create-vocab-all

The sparse data is represented as smile.util.SparseArray.

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	:scicloj.ml.smile.metamorph/bow->sparse-vocabulary

bow->tfidf clj

(bow->tfidf bow-column tfidf-column)

Calculates the tfidf score from bag-of-words (as token frequency maps) in column bow-column and stores them in a new column tfid-column as maps of token->tfidf-score.

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	none

count-vectorize clj

(count-vectorize text-col bow-col)

(count-vectorize text-col bow-col options)

Transforms the text column text-col into a map of token frequencies in column bow-col

options can be any of

text->bow-fn A functions which takes as input a

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	none