scicloj.ml.smile.metamorph — scicloj/scicloj.ml.smile 7.4

bow->something-sparse clj

(bow->something-sparse bow-col indices-col bow->sparse-fn options)

Converts a bag-of-word column bow-col to a sparse data column indices-col. The exact transformation to the sparse representtaion is given by bow->sparse-fn

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	:scicloj.ml.smile.metamorph/bow->sparse-vocabulary

bow->sparse-array clj

(bow->sparse-array bow-col indices-col)

(bow->sparse-array bow-col indices-col options)

Converts a bag-of-word column bow-col to sparse indices column indices-col, as needed by the Maxent model. Options can be of:

create-vocab-fn A function which converts the bow map to a list of tokens. Defaults to scicloj.ml.smile.nlp/create-vocab-all

The sparse data is represented as primitive int arrays, of which entries are the indices against the vocabulary of the present tokens.

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	:scicloj.ml.smile.metamorph/bow->sparse-vocabulary

bow->SparseArray clj

(bow->SparseArray bow-col indices-col)

(bow->SparseArray bow-col indices-col options)

Converts a bag-of-word column bow-col to sparse indices column indices-col, as needed by the discrete naive bayes model.

Options can be of:

create-vocab-fn A function which converts the bow map to a list of tokens. Defaults to scicloj.ml.smile.nlp/create-vocab-all

The sparse data is represented as smile.util.SparseArray.

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	:scicloj.ml.smile.metamorph/bow->sparse-vocabulary

bow->tfidf clj

(bow->tfidf bow-column tfidf-column options)

Calculates the tfidf score from bag-of-words (as token frequency maps) in column bow-column and stores them in a new column tfid-column as maps of token->tfidf-score.

It calculates a global term-frequency map in :fit and reuses it in :transform

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	none

count-vectorize clj

(count-vectorize text-col bow-col)

(count-vectorize text-col bow-col options)

Transforms the text column text-col into a map of token frequencies in column bow-col

options can be any of

• text->bow-fn A functions which takes as input a text as string and options. The default is nlp/default-text->bow

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	none

tfidf->dense-array clj

(tfidf->dense-array tfidf-column dense-column)

Converts the sparse tfidf map based representation into dense double arrays

metamorph	.
Behaviour in mode :fit	normal
Behaviour in mode :transform	normal
Reads keys from ctx	none
Writes keys to ctx	none