criterium.domain.analysis — org.hugoduncan/criterium 0.5.245-ALPHA

Analysis functions for domain data.

Provides functions for extracting metrics, comparing across dimensions, fitting complexity models, and composable analysis pipelines.

->domain-analyse clj

(->domain-analyse analyse-plan)

Creates a composite analysis function from a sequence of analysis specs.

Each spec is either a keyword/symbol to resolve a function, or a vector with a keyword/symbol first element followed by an options map.

Analysis functions are resolved from the criterium.domain.analysis namespace. They are composed in sequence, each taking and returning a data-map.

Example specs: [[:domain-extract-fn {:metric-path [...]}] [:domain-regression-fn {:axis :n}]]

Returns a function that takes a data-map and returns the analyzed data-map.

->domain-view clj

(->domain-view view-plan)

Creates a composite view function from a sequence of view specs.

Each spec is either a keyword/symbol to resolve a function, or a vector with a keyword/symbol first element followed by an options map.

View functions are resolved from the criterium.view namespace. They are called in order with the data-map, producing side effects.

Example specs: [[:domain-extract-table {}] [:domain-regression {}]]

Returns a function that takes a viewer keyword and data-map, returns the viewer output (e.g., kindly fragment for :kindly viewer).

analyse-domain clj

(analyse-domain domain-plan domain)

Analyze a domain using a domain plan.

The domain plan is a map with: :analyse - Vector of analysis specs resolved from criterium.domain.analysis :view - Vector of view specs resolved from criterium.view :viewer - Keyword specifying output format (:print, :portal, :kindly, :none) If not specified, uses the default viewer from criterium.bench/set-default-viewer! :return-value - Path to extract from result data-map (default varies by viewer: [:viewer :output] for :kindly, nil for others which returns the full data-map)

Returns the value at :return-value path, or the full data-map if not specified.

Example: (analyse-domain domain-plans/complexity-analysis my-domain)

(analyse-domain (options->domain-plan domain-plans/complexity-analysis :viewer :portal) my-domain)

compare-by clj

(compare-by domain axis-key metric-path)

(compare-by domain axis-key metric-path {:keys [metric-ids with-error-bounds]})

Compare metric values across an axis dimension. Returns a domain-comparison result showing how the metric varies across axis values.

axis-key is the dimension to compare across. metric-path is [stats-id metric-id value-key] as used by extract.

When metric-path is nil, discovers all quantitative metrics from the first run's :stats :metrics-defs, similar to extract. In multi-metric mode, the domain must have an :implementations key.

By default, comparison uses the bootstrapped median (quantile 0.5) when available, falling back to mean from stats when bootstrap stats are unavailable. When a metric-path is provided that explicitly requests :mean (e.g., [:stats :elapsed-time :mean]), the mean is extracted directly.

Options: :metric-ids - When metric-path is nil, filter to only these metric-ids. :with-error-bounds - When true, extracts error bounds for each value. Prefers bootstrap CI from quantile 0.5 when available, falls back to ±3σ from stats. Values become maps with :value, :lower, and :upper keys.

In multi-metric mode, bootstrap quantile statistics (:median, :p10, :p90, :ci-lower, :ci-upper) are automatically included when available. This enables box plot visualization alongside the default bar charts.

Single metric example: (compare-by domain :impl [:stats :elapsed-time :mean]) ;; => {:type :criterium/domain-comparison ;; :axis :impl ;; :metric [:stats :elapsed-time :mean] ;; :data {:foo [{:coord {:n 100 :impl :foo} :value 1.2e-6} ...] ;; :bar [{:coord {:n 100 :impl :bar} :value 2.3e-6} ...]}}

Multi-metric example: (compare-by domain :impl nil) ;; => {:type :criterium/domain-comparison ;; :axis :impl ;; :implementations [:foo :bar] ;; :metrics {:elapsed-time {:metric [:stats :elapsed-time :mean] ;; :data {:foo [...] :bar [...]}} ;; :thread-allocation {:metric [...] :data {...}}}}

compute-aic clj

(compute-aic rss n k)

Compute AICc (Akaike Information Criterion with small-sample correction).

AICc = n*ln(RSS/n) + 2k + (2k(k+1))/(n-k-1)

Where n = sample size, k = number of parameters, RSS = residual sum of squares.

Returns nil if n <= k+1 (insufficient data for the correction term).

compute-bic clj

(compute-bic rss n k)

Compute BIC (Bayesian Information Criterion).

BIC = nln(RSS/n) + kln(n)

Where n = sample size, k = number of parameters, RSS = residual sum of squares.

Returns nil if n = 0.

default-complexity-models clj

Default complexity models for regression fitting.

Simple models use :transform to map input size n to a single predictor.

Composite models use :transforms for multiple predictors (e.g., n*log(n) + n).

Each model also includes: :predict-fn - Takes coefficients map → (fn [x] predicted-y) :equation-fn - Takes coefficients map → formatted equation string

domain-compare-fn clj

(domain-compare-fn)

(domain-compare-fn {:keys [id domain-id axis-key metric-path metric-ids
                           with-error-bounds]})

Returns a function that compares metric values across an axis in a data-map.

Parameters: opts - Map with keys: :id - Key for result in output (default: :comparison) :domain-id - Key for source domain in input (default: :domain) :axis-key - Dimension key to compare across :metric-path - Vector path to metric, e.g. [:stats :elapsed-time :mean]. When nil, extracts all quantitative metrics. :metric-ids - When metric-path is nil, filter to these metric-ids. E.g., [:elapsed-time :thread-allocation]. :with-error-bounds - When true and extracting :mean values, also extracts error bounds (±3σ) for each value.

In multi-metric mode, bootstrap quantile statistics (median, p10, p90, CI) are automatically included when available, enabling box plot visualization.

The returned function:

• Takes a data-map containing a domain under :domain-id
• Returns the data-map with a domain-comparison result added under :id

Example - single metric: (-> {:domain my-domain} ((domain-compare-fn {:id :impl-vs-time :axis-key :impl :metric-path [:stats :elapsed-time :mean]})))

Example - all metrics: (-> {:domain my-domain} ((domain-compare-fn {:id :impl-comparison :axis-key :impl})))

domain-extract-fn clj

(domain-extract-fn)

(domain-extract-fn {:keys [id domain-id metric-path metric-ids
                           with-error-bounds]})

Returns a function that extracts metric values from a domain in a data-map.

Parameters: opts - Map with keys: :id - Key for result in output (default: :extract) :domain-id - Key for source domain in input (default: :domain) :metric-path - Vector path to metric, e.g. [:stats :elapsed-time :mean]. When nil, extracts all quantitative metrics. :metric-ids - When metric-path is nil, filter to these metric-ids. e.g., [:elapsed-time :thread-allocation]. :with-error-bounds - When true and extracting :mean values, also extracts error bounds (±3σ) for each value.

The returned function:

• Takes a data-map containing a domain under :domain-id
• Returns the data-map with a domain-extract result added under :id

Example - single metric: (-> {:domain my-domain} ((domain-extract-fn {:id :mean :metric-path [:stats :elapsed-time :mean]})))

Example - all metrics: (-> {:domain my-domain} ((domain-extract-fn {:id :metrics})))

domain-group-by-fn clj

(domain-group-by-fn)

(domain-group-by-fn {:keys [id domain-id axis-key]})

Returns a function that groups a domain by axis key in a data-map.

Parameters: opts - Map with keys: :id - Key for result in output (default: :grouped) :domain-id - Key for source domain in input (default: :domain) :axis-key - Dimension key to group by

The returned function:

• Takes a data-map containing a domain under :domain-id
• Returns the data-map with a domain-grouped result added under :id

Example: (-> {:domain my-domain} ((domain-group-by-fn {:id :by-impl :axis-key :impl}))) ;; => {:domain my-domain ;; :by-impl {:type :criterium/domain-grouped ...}}

domain-log-log-fn clj

(domain-log-log-fn)

(domain-log-log-fn {:keys [id extract-id axis]})

Returns a function that fits log-log regression to a domain extract.

Parameters: opts - Map with keys: :id - Key for result in output (default: :log-log) :extract-id - Key for source domain-extract in input (default: :extract) :axis - Coordinate key to use for x-values (required, e.g., :n)

The returned function:

• Takes a data-map containing a domain-extract under :extract-id
• Returns the data-map with a domain-log-log-regression result added under :id

Example: (-> {:domain my-domain} ((domain-extract-fn {:id :extract :metric-path [:stats :elapsed-time :mean]})) ((domain-log-log-fn {:id :log-log :axis :n}))) ;; => {:domain my-domain ;; :extract {:type :criterium/domain-extract ...} ;; :log-log {:type :criterium/domain-log-log-regression ...}}

domain-regression-fn clj

(domain-regression-fn)

(domain-regression-fn {:keys [id extract-id axis models selection-method]})

Returns a function that fits complexity models to a domain extract.

Parameters: opts - Map with keys: :id - Key for result in output (default: :regression) :extract-id - Key for source domain-extract in input (default: :extract) :axis - Coordinate key to use for x-values (required, e.g., :n) :models - Map of model definitions, or nil for defaults :selection-method - Method for selecting best model: :aic (default) - lowest AICc :bic - lowest BIC :r-squared - highest R²

The returned function:

• Takes a data-map containing a domain-extract under :extract-id
• Returns the data-map with a domain-regression result added under :id

Example: (-> {:domain my-domain} ((domain-extract-fn {:id :extract :metric-path [:stats :elapsed-time :mean]})) ((domain-regression-fn {:id :scaling :axis :n}))) ;; => {:domain my-domain ;; :extract {:type :criterium/domain-extract ...} ;; :scaling {:type :criterium/domain-regression ...}}

extract clj

(extract domain)

(extract domain metric-path)

(extract domain metric-path {:keys [with-error-bounds metric-ids]})

Extract metric values from all runs in a domain. Returns a domain-extract result with a :metrics map.

When metric-path is provided, extracts that single metric. When metric-path is nil or omitted, extracts all quantitative metrics discovered from the first run's :stats :metrics-defs.

metric-path is a vector like [:stats :elapsed-time :mean].

By default, extraction uses the bootstrapped median (quantile 0.5) when available, falling back to mean from stats when bootstrap stats are unavailable. When a metric-path is provided that explicitly requests :mean (e.g., [:stats :elapsed-time :mean]), the mean is extracted directly.

Options: :with-error-bounds - When true, extracts error bounds for each value. Prefers bootstrap CI from quantile 0.5 when available, falls back to ±3σ from stats. Values become maps with 108 :value, :lower, and :upper keys. :metric-ids - When metric-path is nil, filter to only these metric-ids (e.g., [:elapsed-time :thread-allocation]). If nil, extracts all quantitative metrics.

For stats paths, transforms are applied to convert raw values to their display form.

Returns nil for value if the metric is missing from a run.

If the source domain has multiple implementations, :impl-axis and :implementations are preserved in the extract result for use by downstream analysis functions.

Example - single metric: (extract domain [:stats :elapsed-time :mean]) ;; => {:type :criterium/domain-extract ;; :metrics {:elapsed-time {:metric [:stats :elapsed-time :mean] ;; :data [[{:n 100} 1.23e-6] ...]}}}

Example - all metrics (uses bootstrapped median): (extract domain) ;; => {:type :criterium/domain-extract ;; :metrics {:elapsed-time ;; {:metric [:stats :elapsed-time :median] :data [...]} ;; :thread-allocation ;; {:metric [:stats :thread-allocation :median] ;; :data [...]}}}

fit-complexity clj

(fit-complexity extract axis)

(fit-complexity extract axis opts)

Fit complexity models to domain extract data.

Returns a domain-regression result identifying how metrics scale with input size.

extract is a domain-extract result (with :metrics map). axis is the coordinate key to use for x-values (e.g., :n for input size).

Options (third argument can be a models map for backward compatibility, or an options map): :models - Map of model-id to {:transform fn :label string}, or nil for defaults :selection-method - Method for selecting best model: :aic (default) - lowest AICc :bic - lowest BIC :r-squared - highest R²

Filters out data points with nil values before fitting. Selects best-fit model using the specified selection method, preferring simpler models when values are essentially equal.

When the extract has multiple implementations (count of :implementations > 1), data is grouped by implementation and models are fit separately for each. The result includes :impl-axis key and regression data in :by-impl maps.

Example - single implementation: (fit-complexity extract :n) ;; => {:type :criterium/domain-regression ;; :axis :n ;; :regressions ;; {:elapsed-time {:metric [:stats :elapsed-time :mean] ;; :models [{:id :linear :r-squared 0.98 ...}] ;; :best-fit :linear}}}

Example - with selection method: (fit-complexity extract :n {:selection-method :bic})

Example - multiple implementations: (fit-complexity extract-with-impls :n) ;; => {:type :criterium/domain-regression ;; :axis :n ;; :impl-axis :impl ;; :implementations [:vec :list] ;; :regressions ;; {:elapsed-time {:metric [:stats :elapsed-time :mean] ;; :by-impl {:vec {:models [...] :best-fit :linear} ;; :list {:models [...] :best-fit :quadratic}}}}}

fit-log-log clj

(fit-log-log extract axis)

Fit log-log linear regression to domain extract data.

Returns a domain-log-log-regression result with log-transformed data and regression statistics for each metric.

extract is a domain-extract result (with :metrics map). axis is the coordinate key to use for x-values (e.g., :n for input size).

The log-log regression fits: log(y) = slope * log(x) + intercept The slope directly indicates complexity class:

• slope ≈ 1 → O(n)
• slope ≈ 2 → O(n²)

Filters out data points with nil or non-positive values before fitting.

When the extract has multiple implementations, data is grouped by implementation and regression is fit separately for each.

Example - single implementation: (fit-log-log extract :n) ;; => {:type :criterium/domain-log-log-regression ;; :axis :n ;; :regressions {:elapsed-time {:metric [:stats :elapsed-time :mean] ;; :slope 1.02 ;; :intercept -15.3 ;; :r-squared 0.998 ;; :log-xs [...] ;; :log-ys [...] ;; :xs [...] ;; :ys [...] ;; :residuals [...]}}}

Example - multiple implementations: (fit-log-log extract-with-impls :n) ;; => {:type :criterium/domain-log-log-regression ;; :axis :n ;; :impl-axis :impl ;; :implementations [:vec :list] ;; :regressions {:elapsed-time {:metric [:stats :elapsed-time :mean] ;; :by-impl {:vec {:slope 1.0 ...} ;; :list {:slope 2.0 ...}}}}}

group-by-axis clj

(group-by-axis domain axis-key)

Partition domain runs by values of an axis key. Returns a domain-grouped result containing sub-domains by axis value.

Runs with map coordinates are grouped by the value of axis-key. Runs without the axis-key (including keyword coordinates) are grouped under nil.

Example: (group-by-axis domain :impl) ;; => {:type :criterium/domain-grouped ;; :axis :impl ;; :data {:foo <domain with :impl :foo runs> ;; :bar <domain with :impl :bar runs> ;; nil <domain with runs lacking :impl>}}

log-log-regression clj

(log-log-regression xs ys)

Perform linear regression on log-transformed data.

Transforms (x, y) to (log(x), log(y)) and fits a linear model: log(y) = slope * log(x) + intercept

The slope directly indicates algorithmic complexity:

• slope ≈ 0 → O(1)
• slope ≈ 0.5 → O(√n)
• slope ≈ 1 → O(n)
• slope ≈ 1.5 → O(n√n)
• slope ≈ 2 → O(n²)

Returns: {:slope - complexity exponent :intercept - log-space intercept :r-squared - coefficient of determination in log-log space :log-xs - log-transformed x values :log-ys - log-transformed y values :residuals - residuals in log space :predict-fn - (fn [x] predicted-y) in original space}

options->domain-plan clj

(options->domain-plan base-plan & {:as options})

Merge options into a base domain plan.

Takes a base plan map and keyword/value pairs to override specific fields.

Example: (options->domain-plan domain-plans/complexity-analysis :viewer :portal)