fastester.measure — com.sagevisuals/fastester 0-SNAPSHOT1

Define and run benchmarks that objectively measure a function's evaluation time.

The general idea is to run the benchmark suite once per release. Any performance improvement/regression is objectively measured and included in the changelog/release notes.

The Criterium library does all the actual performance measurements.

See fastester.display for utilities that generate an html document with charts and tables that communicate those results.

*lightning-benchmark-opts* clj

Criterium options with extremely minimal samples, etc. Use only for quick, proof-of-concept runs.

Example:

  (criterium.core/benchmark (+ 1 2) *lightning-benchmark-opts*)

See also criterium.core/*default-benchmark-opts* and criterium.core/*default-quick-bench-opts*.

range-pow-10 clj

(range-pow-10 i)

Given integer i, returns a sequence of [10^0...10^i], inclusive. Useful for supplying to benchmarking functions.

Examples:

(range-pow-10 3) ;; => (1 10 100 1000)
(range-pow-10 6) ;; => (1 10 100 1000 10000 100000 1000000)

See also range-pow-2.

range-pow-2 clj

(range-pow-2 i)

Given integer i, returns a sequence of [2^0...2^i], inclusive. Useful for supplying to benchmarking functions.

Example:

(range-pow-2 8) ;; (1 2 4 8 16 32 64 128 256)

See also range-pow-10.

run-benchmarks clj

(run-benchmarks)

(run-benchmarks explicit-options-filename)

Execute benchmarks associated to option key :benchmarks, saving results to a version-specific directory. If explicit-options-filename is not supplied, defaults to ./resources/fastester_options.edn.

Upon invocation, reads options file and any benchmarks definition namespace files from the disk. Ensure that the state of the file on disk and that same file opened in a REPL-attached editor is what you intend.

If option :parallel? is true, runs tests in parallel. Warning: Running tests in parallel results in inconsistent time measurements. Use :parallel? true only for sanity-checking the return values of (f n), not for final performance measurements.

run-manual-benchmark clj macro

(run-manual-benchmark fexpr arg thoroughness)

Given 1-arity S-expression fexpr representing a function and one argument arg, run one benchmark test using keyword thoroughness to designate the Criterium options, sending result to *out*.

thoroughness is one of :default, :quick, or :lightning.

Example:

(run-manual-benchmark (fn [n] (+ n n)) 9 :lightning)

See run-one-defined-benchmark and *lightning-benchmark-opts*.

run-one-defined-benchmark clj

(run-one-defined-benchmark benchmark thoroughness)

Given defined benchmark, an unquoted symbol, and keyword thoroughness that designates the Criterium options, runs a benchmark for each defined argument. Returns a hashmap whose keys are arguments n, associated to values that are the benchmark results for that n.

thoroughness is one of :default, :quick, or :lightning.

Example:

;; define a benchmark
(defbench my-bench "my-group" (fn [x] (inc x)) [97 98 99])

(run-one-defined-benchmark my-bench :lightning)
;; => {97 {:mean ... :var ...}
;;     98 {:mean ... :var ...}
;;     99 {:mean ... :var ...}}

The above example returns a hashmap with three key+vals: keys 97, 98, and 99, each associated with its respective benchmark result.

See also run-manual-benchmark and *lightning-benchmark-opts*.

run-save-benchmark clj

(run-save-benchmark ver name namespace group f fexpr arg idx opts)

Given string ver, symbol name, namespace namespace, string group, function object f, function S-expression fexpr, test argument arg, index integer idx, and option hashmap opts, measures the evaluation time according to the option's :testing-thoroughness for Criterium benchmark settings and saves results to file system.

Example:

(run-save-benchmark "77-SNAPSHOT7"
                    'mult-two-nums
                    'benchmark-namespace
                    "adding --- stuff"
                    (fn [n] (* n n))
                    '(fn [n] (* n n))
                    22
                    5
                    options)

See also run-manual-benchmark and run-one-defined-benchmark for utilities to quickly benchmark an expression.