zero-one.geni.ml — zero.one/geni 0.0.31

aft-survival-regression clj

(aft-survival-regression params)

Fit a parametric survival regression model named accelerated failure time (AFT) model (see Accelerated failure time model (Wikipedia)) based on the Weibull distribution of the survival time.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/AFTSurvivalRegression.html Timestamp: 2020-10-02T14:21:20.345Z

als clj

(als params)

alternating-least-squares clj

approx-nearest-neighbours clj

(approx-nearest-neighbours dataset model key-v n-nearest)

(approx-nearest-neighbours dataset model key-v n-nearest dist-col)

approx-similarity-join clj

(approx-similarity-join dataset-a dataset-b model threshold)

(approx-similarity-join dataset-a dataset-b model threshold dist-col)

association-rules clj

(association-rules model)

best-model clj

(best-model model)

binariser clj

(binariser params)

binarizer clj

binary-classification-evaluator clj

(binary-classification-evaluator params)

binary-summary clj

(binary-summary model)

bisecting-k-means clj

(bisecting-k-means params)

boundaries clj

(boundaries model)

bucketed-random-projection-lsh clj

(bucketed-random-projection-lsh params)

bucketiser clj

(bucketiser params)

bucketizer clj

category-maps clj

(category-maps model)

category-sizes clj

(category-sizes model)

chi-sq-selector clj

(chi-sq-selector params)

chi-square-test clj

(chi-square-test dataframe features-col label-col)

cluster-centers clj

(cluster-centers model)

clustering-evaluator clj

(clustering-evaluator params)

coefficient-matrix clj

(coefficient-matrix model)

coefficients clj

(coefficients model)

corr clj multimethod

count-vectoriser clj

(count-vectoriser params)

count-vectorizer clj

cross-validator clj

(cross-validator {:keys [estimator evaluator estimator-param-maps num-folds seed
                         parallelism]})

dct clj

decision-tree-classifier clj

(decision-tree-classifier params)

decision-tree-regressor clj

(decision-tree-regressor params)

Decision tree learning algorithm for regression. It supports both continuous and categorical features.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.html Timestamp: 2020-10-02T14:21:20.720Z

depth clj

(depth model)

describe-topics clj

discrete-cosine-transform clj

(discrete-cosine-transform params)

distributed? clj

elementwise-product clj

(elementwise-product params)

estimated-doc-concentration clj

(estimated-doc-concentration model)

evaluate clj

(evaluate dataframe evaluator)

feature-hasher clj

(feature-hasher params)

feature-importances clj

(feature-importances model)

features-col clj

find-frequent-sequential-patterns clj

(find-frequent-sequential-patterns dataset prefix-span)

find-patterns clj

fit clj

(fit dataframe estimator)

fm-classifier clj

(fm-classifier params)

fm-regressor clj

(fm-regressor params)

Factorization Machines learning algorithm for regression. It supports normal gradient descent and AdamW solver.The implementation is based upon:

S. Rendle. "Factorization machines" 2010.FM is able to estimate interactions even in problems with huge sparsity (like advertising and recommendation system). FM formula is:

$$ \begin{align} y = w_0 + \sum\limits^n_{i-1} w_i x_i + \sum\limits^n_{i=1} \sum\limits^n_{j=i+1} \langle v_i, v_j \rangle x_i x_j \end{align} $$

First two terms denote global bias and linear term (as same as linear regression), and last term denotes pairwise interactions term. v_i describes the i-th variable with k factors.FM regression model uses MSE loss which can be solved by gradient descent method, and regularization terms like L2 are usually added to the loss function to prevent overfitting.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/FMRegressor.html Timestamp: 2020-10-02T14:21:21.102Z

fp-growth clj

(fp-growth params)

freq-itemsets clj

frequent-item-sets clj

(frequent-item-sets model)

frequent-pattern-growth clj

gaussian-mixture clj

(gaussian-mixture params)

gaussians-df clj

(gaussians-df model)

gbt-classifier clj

(gbt-classifier params)

gbt-regressor clj

(gbt-regressor params)

Gradient-Boosted Trees (GBTs) learning algorithm for regression. It supports both continuous and categorical features.The implementation is based upon: J.H. Friedman. "Stochastic Gradient Boosting." 1999.Notes on Gradient Boosting vs. TreeBoost:This implementation is for Stochastic Gradient Boosting, not for TreeBoost.Both algorithms learn tree ensembles by minimizing loss functions.TreeBoost (Friedman, 1999) additionally modifies the outputs at tree leaf nodes based on the loss function, whereas the original gradient boosting method does not.When the loss is SquaredError, these methods give the same result, but they could differ for other loss functions.We expect to implement TreeBoost in the future: [https://issues.apache.org/jira/browse/SPARK-4240]

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/GBTRegressor.html Timestamp: 2020-10-02T14:21:21.485Z

generalised-linear-regression clj

(generalised-linear-regression params)

Fit a Generalized Linear Model (see Generalized linear model (Wikipedia)) specified by giving a symbolic description of the linear predictor (link function) and a description of the error distribution (family). It supports "gaussian", "binomial", "poisson", "gamma" and "tweedie" as family. Valid link functions for each family is listed below. The first link function of each family is the default one."gaussian" : "identity", "log", "inverse""binomial" : "logit", "probit", "cloglog""poisson" : "log", "identity", "sqrt""gamma" : "inverse", "identity", "log""tweedie" : power link function specified through "linkPower". The default link power in the tweedie family is 1 - variancePower.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.html Timestamp: 2020-10-02T14:21:21.946Z

generalized-linear-regression clj

(generalized-linear-regression params)

Fit a Generalized Linear Model (see Generalized linear model (Wikipedia)) specified by giving a symbolic description of the linear predictor (link function) and a description of the error distribution (family). It supports "gaussian", "binomial", "poisson", "gamma" and "tweedie" as family. Valid link functions for each family is listed below. The first link function of each family is the default one."gaussian" : "identity", "log", "inverse""binomial" : "logit", "probit", "cloglog""poisson" : "log", "identity", "sqrt""gamma" : "inverse", "identity", "log""tweedie" : power link function specified through "linkPower". The default link power in the tweedie family is 1 - variancePower.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.html Timestamp: 2020-10-02T14:21:21.946Z

get-features-col clj

(get-features-col model)

get-input-col clj

(get-input-col model)

get-input-cols clj

(get-input-cols model)

get-label-col clj

(get-label-col model)

get-num-trees clj

(get-num-trees model)

get-output-col clj

(get-output-col model)

get-output-cols clj

(get-output-cols model)

get-prediction-col clj

(get-prediction-col model)

get-probability-col clj

(get-probability-col model)

get-raw-prediction-col clj

(get-raw-prediction-col model)

get-size clj

(get-size model)

get-thresholds clj

(get-thresholds model)

glm clj

(glm params)

Fit a Generalized Linear Model (see Generalized linear model (Wikipedia)) specified by giving a symbolic description of the linear predictor (link function) and a description of the error distribution (family). It supports "gaussian", "binomial", "poisson", "gamma" and "tweedie" as family. Valid link functions for each family is listed below. The first link function of each family is the default one."gaussian" : "identity", "log", "inverse""binomial" : "logit", "probit", "cloglog""poisson" : "log", "identity", "sqrt""gamma" : "inverse", "identity", "log""tweedie" : power link function specified through "linkPower". The default link power in the tweedie family is 1 - variancePower.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.html Timestamp: 2020-10-02T14:21:21.946Z

gmm clj

hashing-tf clj

(hashing-tf params)

idf clj

(idf params)

idf-vector clj

(idf-vector model)

imputer clj

(imputer params)

index-to-string clj

(index-to-string params)

input-col clj

input-cols clj

interaction clj

(interaction params)

intercept clj

(intercept model)

intercept-vector clj

(intercept-vector model)

is-distributed clj

(is-distributed model)

isotonic-regression clj

(isotonic-regression params)

Isotonic regression.Currently implemented using parallelized pool adjacent violators algorithm. Only univariate (single feature) algorithm supported.Uses org.apache.spark.mllib.regression.IsotonicRegression.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/IsotonicRegression.html Timestamp: 2020-10-02T14:21:22.305Z

item-factors clj

(item-factors model)

k-means clj

(k-means params)

kolmogorov-smirnov-test clj

(kolmogorov-smirnov-test dataframe sample-col dist-name params)

label-col clj

labels clj

(labels model)

latent-dirichlet-allocation clj

lda clj

(lda params)

linear-regression clj

(linear-regression params)

Linear regression.The learning objective is to minimize the specified loss function, with regularization. This supports two kinds of loss:squaredError (a.k.a squared loss)huber (a hybrid of squared error for relatively small errors and absolute error for relatively large ones, and we estimate the scale parameter from training data)This supports multiple types of regularization:none (a.k.a. ordinary least squares)L2 (ridge regression)L1 (Lasso)L2 + L1 (elastic net)The squared error objective function is: $$ \begin{align} \min_{w}\frac{1}{2n}{\sum_{i=1}^n(X_{i}w - y_{i})^{2} + \lambda\left[\frac{1-\alpha}{2}{||w||{2}}^{2} + \alpha{||w||{1}}\right]} \end{align} $$ The huber objective function is: $$ \begin{align} \min_{w, \sigma}\frac{1}{2n}{\sum_{i=1}^n\left(\sigma + H_m\left(\frac{X_{i}w - y_{i}}{\sigma}\right)\sigma\right) + \frac{1}{2}\lambda {||w||_2}^2} \end{align} $$ where $$ \begin{align} H_m(z) = \begin{cases} z^2, & \text {if } |z| < \epsilon, \ 2\epsilon|z| - \epsilon^2, & \text{otherwise} \end{cases} \end{align} $$ Note: Fitting with huber loss only supports none and L2 regularization.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/LinearRegression.html Timestamp: 2020-10-02T14:21:22.713Z

linear-svc clj

(linear-svc params)

load-method clj

(load-method cls)

load-method? clj

(load-method? method)

log-likelihood clj

(log-likelihood dataset model)

log-perplexity clj

(log-perplexity dataset model)

logistic-regression clj

(logistic-regression params)

max-abs clj

(max-abs model)

max-abs-scaler clj

(max-abs-scaler params)

mean clj

(mean model)

min-hash-lsh clj

(min-hash-lsh params)

min-max-scaler clj

(min-max-scaler params)

mlp-classifier clj

(mlp-classifier params)

multiclass-classification-evaluator clj

(multiclass-classification-evaluator params)

multilabel-classification-evaluator clj

(multilabel-classification-evaluator params)

multilayer-perceptron-classifier clj

n-gram clj

(n-gram params)

naive-bayes clj

(naive-bayes params)

normaliser clj

(normaliser params)

normalizer clj

num-classes clj

(num-classes model)

num-features clj

(num-features model)

num-nodes clj

(num-nodes model)

one-hot-encoder clj

(one-hot-encoder params)

one-vs-rest clj

(one-vs-rest params)

original-max clj

(original-max model)

original-min clj

(original-min model)

output-col clj

output-cols clj

param-grid clj

(param-grid grids)

params clj

(params stage)

pc clj

(pc model)

pca clj

(pca params)

pi clj

(pi model)

pipeline clj

(pipeline & stages)

polynomial-expansion clj

(polynomial-expansion params)

power-iteration-clustering clj

(power-iteration-clustering params)

prediction-col clj

prefix-span clj

(prefix-span params)

principal-components clj

probability-col clj

quantile-discretiser clj

(quantile-discretiser params)

quantile-discretizer clj

random-forest-classifier clj

(random-forest-classifier params)

random-forest-regressor clj

(random-forest-regressor params)

Random Forest learning algorithm for regression. It supports both continuous and categorical features.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/ml/regression/RandomForestRegressor.html Timestamp: 2020-10-02T14:21:23.298Z

ranking-evaluator clj

(ranking-evaluator params)

raw-prediction-col clj

read-stage! clj

(read-stage! model-cls path)

recommend-for-all-items clj

(recommend-for-all-items model num-users)

recommend-for-all-users clj

(recommend-for-all-users model num-items)

recommend-for-item-subset clj

(recommend-for-item-subset model items-df num-users)

recommend-for-user-subset clj

(recommend-for-user-subset model users-df num-items)

recommend-items clj

(recommend-items model num-items)

(recommend-items model users-df num-items)

recommend-users clj

(recommend-users model num-users)

(recommend-users model items-df num-users)

regex-tokeniser clj

(regex-tokeniser params)

regex-tokenizer clj

regression-evaluator clj

(regression-evaluator params)

robust-scaler clj

(robust-scaler params)

root-node clj

(root-node model)

scale clj

(scale model)

sql-transformer clj

(sql-transformer params)

stages clj

(stages model)

standard-scaler clj

(standard-scaler params)

std clj

(std model)

stop-words-remover clj

(stop-words-remover params)

string-indexer clj

(string-indexer params)

summary clj

(summary model)

supported-optimisers clj

supported-optimizers clj

(supported-optimizers model)

surrogate-df clj

(surrogate-df model)

theta clj

(theta model)

thresholds clj

tokeniser clj

(tokeniser params)

tokenizer clj

total-num-nodes clj

(total-num-nodes model)

train-validation-split clj

(train-validation-split {:keys [estimator evaluator estimator-param-maps seed
                                parallelism]})

transform clj

(transform dataframe transformer)

tree-weights clj

(tree-weights model)

trees clj

(trees model)

uid clj

(uid model)

user-factors clj

(user-factors model)

vector->array clj

vector-assembler clj

(vector-assembler params)

vector-indexer clj

(vector-indexer params)

vector-size-hint clj

(vector-size-hint params)

vector-to-array clj

(vector-to-array expr)

(vector-to-array expr dtype)

vocab-size clj

(vocab-size model)

vocabulary clj

(vocabulary model)

weights clj

(weights model)

word2vec clj

(word2vec params)

write-native-model! clj

(write-native-model! model path)

write-stage! clj

(write-stage! stage path)

(write-stage! stage path options)

xgboost-classifier clj

(xgboost-classifier params)

xgboost-regressor clj

(xgboost-regressor params)