fastmath.classification — generateme/fastmath 1.4.0-SNAPSHOT

Classification algorithms.

Input data

• features - sequence of sequences of numbers
• categories - sequence of any values

Workflow

• create classifier with parameters
• cross validate [[cv]]
• repeat or [[predict]]
• to validate model against test data, call validate

Classifier parameters are map of values specific for given algorithm. Check documentation in backend library to find documentation. Classifier can be retrained using [[train]]. New instance will be created.

Classifier training is delayed to the actual use. To force training, call [[train]] or [[predict]].

Implementation notes

• only doubles as input data
• categories can be any type

Cross validation

Every classifier exposes it's own cross validation method with configuration [[cv]].

~~Additionally three Clojure level methods are defined: [[cv]], [[loocv]] and [[bootstrap]].~~

SMILE

documentation

What is missed:

• other types than doubles (attributes)
• maxent
• Online classifiers
• General Naive Bayes

Native cross validation config is a map with keys:

• :k - number of folds (default: 10)
• :type - type of cross validation, one of :cv (default), :loocv and :bootstrap

Liblinear

https://github.com/bwaldvogel/liblinear-java

Native cross validation expects a number as number of folds (default: 10)

Examples

Iris database is used.

accuracy clj

(accuracy t p)

Calculate accuracy for real and predicted sequences.

activation-functions-list clj

List of activation functions for neural-net.

ada-boost clj

(ada-boost x y)

(ada-boost {:keys [number-of-trees max-nodes]
            :or {number-of-trees 500 max-nodes 2}}
           x
           y)

ada-boost classifier. Backend library: smile

bayes-models-list clj

List of naive-bayes models.

ClassificationProto clj protocol

backend clj

(backend _)

Return name of backend library

cv clj

(cv _)

(cv _ params)

Cross-validation

data-native clj

(data-native _)

(data-native _ native?)

Return data transformed for backend library.

labels clj

(labels _)

Return labels.

model-native clj

(model-native _)

Return trained model as a backend class.

predict clj

(predict _ v)

(predict _ v posteriori?)

Predict class for given vector. With posteriori probabilities when required.

predict-all clj

(predict-all _ vs)

(predict-all _ v posteriori?)

Predict classes for given sequence of vectors. With posteriori probabilities when required.

train clj

(train _)

(train _ x y)

Train another set of data for given classifier or force training already given data.

confusion-map clj

(confusion-map t p)

Create confusion map where keys are pairs of [truth-label prediction-label]

decision-tree clj

(decision-tree x y)

(decision-tree {:keys [max-nodes node-size split-rule]
                :or {max-nodes 100 node-size 1 split-rule :gini}}
               x
               y)

decision-tree classifier. Backend library: smile

error-functions-list clj

List of error functions for neural-net.

fld clj

(fld x y)

(fld {:keys [dimensionality tolerance] :or {dimensionality -1 tolerance 1.0E-4}}
     x
     y)

fld classifier. Backend library: smile

gradient-tree-boost clj

(gradient-tree-boost x y)

(gradient-tree-boost
  {:keys [number-of-trees shrinkage max-nodes subsample]
   :or {number-of-trees 500 shrinkage 0.005 max-nodes 6 subsample 0.7}}
  x
  y)

gradient-tree-boost classifier. Backend library: smile

knn clj

(knn x y)

(knn {:keys [distance k] :or {distance (EuclideanDistance.) k 1}} x y)

knn classifier. Backend library: smile

lda clj

(lda x y)

(lda {:keys [priori tolerance] :or {priori nil tolerance 1.0E-4}} x y)

lda classifier. Backend library: smile

liblinear clj

(liblinear x y)

(liblinear
  {:keys [solver bias C eps max-iters p weights]
   :or
     {solver :l2r-l2loss-svc-dual bias -1 C 1.0 eps 0.01 max-iters 1000 p 0.1}}
  x
  y)

liblinear classifier. Backend library: liblinear

liblinear-solver-list clj

List of liblinear solvers.

logistic-regression clj

(logistic-regression x y)

(logistic-regression {:keys [lambda tolerance max-iterations]
                      :or {lambda 0.0 tolerance 1.0E-5 max-iterations 500}}
                     x
                     y)

logistic-regression classifier. Backend library: smile

multiclass-strategies-list clj

List of multiclass strategies for svm

naive-bayes clj

(naive-bayes x y)

(naive-bayes {:keys [model priori sigma] :or {model :bernoulli sigma 1.0}} x y)

naive-bayes classifier. Backend library: smile

neural-net clj

(neural-net x y)

(neural-net {:keys [error-function activation-function layers learning-rate
                    momentum weight-decay number-of-epochs]
             :or {error-function :cross-entropy
                  learning-rate 0.1
                  momentum 0.0
                  weight-decay 0.0
                  number-of-epochs 25}}
            x
            y)

neural-net classifier. Backend library: smile

qda clj

(qda x y)

(qda {:keys [priori tolerance] :or {priori nil tolerance 1.0E-4}} x y)

qda classifier. Backend library: smile

random-forest clj

(random-forest x y)

(random-forest {:keys [number-of-trees split-rule mtry node-size max-nodes
                       subsample]
                :or {number-of-trees 500
                     split-rule :gini
                     node-size 1
                     max-nodes 100
                     subsample 1.0}}
               x
               y)

random-forest classifier. Backend library: smile

rbf-network clj

(rbf-network x y)

(rbf-network {:keys [distance rbf number-of-basis normalize?]
              :or {distance dist/euclidean number-of-basis 10 normalize? false}}
             x
             y)

rbf-network classifier. Backend library: smile

rda clj

(rda x y)

(rda {:keys [alpha priori tolerance] :or {alpha 0.9 tolerance 1.0E-4}} x y)

rda classifier. Backend library: smile

split-rules-list clj

List of split rules for [[decision tree]] and random-forest

svm clj

(svm x y)

(svm {:keys [kernel c-or-cp cn strategy-for-multiclass class-weights tolerance
             epochs]
      :or {kernel (k/kernel :linear)
           c-or-cp 1.0
           cn 1.0
           strategy-for-multiclass :one-vs-one
           tolerance 0.001
           epochs 2}}
     x
     y)

svm classifier. Backend library: smile

validate clj

(validate model tx ty)

Validate data against trained classifier. Same as [[test]].