taoensso.tukey — com.taoensso/tukey 0.1.0

Simple stats toolkit for Clojure/Script.

Objectives:

• All Clojure/Script implementations.
• Pleasant+consistant API w/ beginner-friendly documentation.
• Start with basic, commonly used utils. Can expand over time.
• Start with short, understandable algorithms. Can optimise over time.

Refererences: [R1] A Computer Method for Calculating Kendall's Tau with Ungrouped Data by William R. Knight, Journal of the American Statistical Association Vol. 61, No. 314, Part 1 (Jun., 1966), pp. 436-439 (4 pages).

[R2] Efficiently Computing Kendall's Tau by Matt Adereth, http://adereth.github.io/blog/2013/10/30/efficiently-computing-kendalls-tau/

arithmetic-mean clj/s

(arithmetic-mean xs)

Returns the arithmetic mean x̄ ∈ ℝ of the given values (n>0).

arithmetic-std-deviation clj/s

(arithmetic-std-deviation xs)

Returns the standard deviation σ ∈ ℝ of the given values (n>1).

combinations clj/s

(combinations {:keys [repeated? ordered? n-take xs] :as opts})

(combinations n-take xs)

(combinations repeated? ordered? n-take xs)

TODO Nice docstring!

covariance clj/s

(covariance xs ys)

Returns the covariance σ(x,y) ∈ ℝ between the given value colls of same size (>1).

factorial clj/s

(factorial n)

Returns factorial n! ∈ ℕ = n∙(n-1)∙(n-2)...3∙2∙1, caching each step. O(n!). May auto-promote to BigInt for Clj.

geometric-mean clj/s

(geometric-mean xs)

TODO Docstring, compare to others

joint-vals clj/s

(joint-vals xs ys)

Returns [[x1 y1] ... [xn yn]] for given joint value colls.

kendall-correlation clj/s

(kendall-correlation xs ys)

(kendall-correlation {:keys [kind impl] :or {kind :tao-b impl :auto}} xs ys)

Returns Kendall rank correlation coefficient tau τ ∈ ℝ[-1,+1] between the given value colls of same size (>0). O(n.log_n) algorithm.

Measures the strength and direction of relationship between xs and ys.

Properties:

• Non-parametric measure.
• Supported value types: continuous, nominal/ordinal.
• Supported relationship: linear, monotonic.

A good general-purpose choice as correlation measure if you're not confident about the probability distribution of your random variables, etc.

Similar to Spearman correlation but:

• [Pro] More robust toward ties.
• [Pro] Easier intuitive interpretation (comparing concordant and discordant pairs of values).
• [Neutral] Less sensitive to a small number of large discrepancies. May be good or bad depending on your needs.
• [Con] O(n.log_n) vs O(n).

Supports 3 kinds:

• :tao-a ; Simplest implementation.
• :tao-b ; Preferred if values may contain ties.
• :tao-c ; Preferred if xs, ys have uneven number of unique values.

See also spearman-correlation, pearson-correlation.

mean clj/s≠

clj

(mean xs)

Returns the arithmetic mean x̄ ∈ ℝ of the given values (n>0).

cljs

mean-abs-deviation clj/s

(mean-abs-deviation xs)

(mean-abs-deviation central-point xs)

Returns the mean absolute deviation MAD ∈ ℝ of the given values (n>0) from a central point (:mean, :median, or arbitrary val).

mean-ranks clj/s

(mean-ranks xs)

Returns ranks ∈ ℝ for the given values, with ties as mean rank: (mean-ranks [0 1 2]) -> [0.0 1.0 2.0] (mean-ranks [0 10 200]) -> [0.0 1.0 2.0] (mean-ranks [0 10 10]) -> [0.0 1.5 1.5]

median clj/s

(median xs)

TODO Docstring, compare to others

min-max clj/s

(min-max xs)

Returns ?[<min> <max>] in:

• O(1) for Sorted types (SortedLongs, SortedDoubles),
• O(n) otherwise.

n-combinations clj/s

(n-combinations {:keys [repeated? ordered? n-take n-xs]})

(n-combinations n-take n-xs)

(n-combinations repeated? ordered? n-take n-xs)

Like combinations, but only returns the cardinality of the result. May auto-promote to BigDecimal for Clj.

n-permutations clj/s

(n-permutations n-xs)

(n-permutations n-take n-xs)

(n-permutations repeated? n-take n-xs)

Like permutations, but only returns the cardinality of the result. May auto-promote to BigDecimal for Clj.

pearson-correlation clj/s

(pearson-correlation xs ys)

Returns Pearson product-moment correlation coefficient (PPMCC) r ∈ ℝ[-1,+1] between the given value colls of same size (>1). O(n).

Measures the strength and direction of relationship between xs and ys.

Properties:

• Parametric measure.
• Supported value types: continuous.
• Supported relationships: linear.

See also spearman-correlation, kendall-correlation.

percentiles clj/s

(percentiles xs)

Returns ?[min p25 p50 p75 p90 p95 p99 max] in:

• O(1) for Sorted types (SortedLongs, SortedDoubles),
• O(n) otherwise.

permutations clj/s

(permutations xs)

(permutations n-take xs)

(permutations repeated? n-take xs)

Like combinations, but always ordered.

reduce-combinations clj/s

(reduce-combinations rfn init n-take xs)

(reduce-combinations xform rfn init n-take xs)

(reduce-combinations xform rfn init {:keys [repeated? ordered?]} n-take xs)

Like combinations but instead of producing and returning a vector of combinations, acts like reduce (or transduce when xform provided).

Since it avoids the construction of a data structure, this can be a lot more efficient if the number of combinations is large, and you don't actually need the data structure.

The xform also offers a lot of flexibility for filtering, early termination, etc.

reduce-permutations clj/s

(reduce-permutations rfn init xs)

(reduce-permutations xform rfn init n-take xs)

(reduce-permutations xform rfn init repeated? n-take xs)

Like reduce-combinations, but always ordered.

sorted-doubles clj/s≠

clj

(sorted-doubles x)

Advanced: returns a SortedDoubles instance for given numerical values, backed by a double array. May be provided to certain utils as a performance optimisation.

cljs

sorted-doubles? clj/s≠

clj

(sorted-doubles? x)

cljs

sorted-longs clj/s≠

clj

(sorted-longs x)

Advanced: returns a SortedLongs instance for given numerical values, backed by a long array. May be provided to certain utils as a performance optimisation.

cljs

sorted-longs? clj/s≠

clj

(sorted-longs? x)

cljs

spearman-correlation clj/s

(spearman-correlation xs xy)

Returns Spearman rank correlation coefficient rho ρ ∈ ℝ[-1,+1] between the given value colls of same size (>1). O(n).

Measures the strength and direction of relationship between xs and ys.

Properties:

• Non-parametric measure.
• Supported value types: continuous, nominal/ordinal.
• Supported relationship: linear, monotonic.

A good general-purpose choice as correlation measure if you're not confident about the probability distribution of your random variables, etc.

Calculated as Pearson correlation of value ranks, with ties as mean rank. Will have a similar coefficient to Pearson for tame data, but is more robust to outliers, and more general (supports any monotonic relationship, not just linear).

See also kendall-correlation, pearson-correlation.

std-deviation clj/s≠

clj

(std-deviation xs)

Returns the standard deviation σ ∈ ℝ of the given values (n>1).

cljs

variance clj/s

(variance xs)

Returns the variance σ² ∈ ℝ of the given values (n>1).