scicloj.metamorph.ml — scicloj/metamorph.ml 0.1.0-alpha2

calc-ctx-with-metric clj

(calc-ctx-with-metric pipeline-fn metric-fn train-ds test-ds)

evaluate-pipeline clj

(evaluate-pipeline pipe-fn train-test-split-seq metric-fn loss-or-accuracy)

evaluate-pipelines clj

(evaluate-pipelines pipe-fn-seq train-test-split-seq metric-fn loss-or-accuracy)

(evaluate-pipelines pipe-fn-seq
                    train-test-split-seq
                    metric-fn
                    loss-or-accuracy
                    n-slices)

Evaluates performance of a seq of metamorph pipelines, which are suposed to have a model as last step, which behaves correctly in mode :fit and :transform It calculates the loss, given as loss-fn of each pipeline in pipeline-fn-seq using all the train-test splits given in train-test-split-seq.

pipe-fn-seq need to be sequence of functions which follow the metamorph approach. They should take as input the metamorph context map, which has the dataset under key :metamorph/data, manipulate it as needed for the transformation pipeline and read and write only to the context as needed. train-test-split-seq need to be a sequence of maps containing the train and test dataset (being tech.ml.dataset) at keys :train and :test. metric-fn Metric function to use. Typically comming from tech.v3.ml.loss loss-or-accuracy If the metric-fn is a loss or accuracy calculation. Can be :loss or :accuracy. n-slices Decides how many results are returned. By default one evaluation results for each pipeline-fn is returned.

In case of a larger number of pipelines, this can become a memory issue. Setting n-slices to lower value, returns max n-slices models, where each model is the best (according to metyric-fn and loss-or-accuracy) of its slice.

This function runs the pipeline in mode :fit and in mode :transform for each pipeline-fn in pipe-fn-seq for each split in train-test-split-seq.

The pipeline-fns need to set as well the ground truth of the target variable into a specific key :scicloj.metamorph.ml/target-ds See here for the simplest way to set this up: https://github.com/behrica/metamorph.ml/blob/main/README.md

predict-on-best-model clj

(predict-on-best-model evaluations new-ds loss-or-accuracy)

slice clj

(slice n coll)

Divide coll into n approximately equal slices.