volley.run-state — com.kirahowe/volley 0.1.0-alpha.2

Folds worker events into the run's state, and derives summaries from it.

The state is a plain map built only by apply-event. Every record is keyed by namespace and var, and timestamps are folded with min/max, so the final state depends only on each worker's own event order and not on how events from different workers interleave.

Event shapes are defined in doc/design.md: the worker events plus what the pool adds ({:event :output ...} for stray output, {:event :worker/exit :code n}), the scheduler's synthetic crash events, and two the engine emits: {:event :file/error :file path :error msg} for a test file whose ns form cannot be read, {:event :run/not-run :namespaces [ns ...] :reason kw} for planned namespaces that never ran, and {:event :run/interrupted :namespaces [ns ...] :reason :fail-fast} for namespaces whose worker was killed part way. Workers also send {:event :worker/error :phase :shutdown :error err} when a shutdown hook throws; it counts as one failure.

A worker crash is recorded once, from the synthetic :ns/error whose error class is volley.WorkerCrashed. :worker/exit only marks the worker as gone.

all-skipped clj

(all-skipped state)

When the run ran no test at all because the filters skipped every one it found, how many they skipped; else nil. Kaocha warns then (kaocha.api/run: "All N tests were skipped").

apply-event clj

(apply-event state event)

Folds one event into state. Events from one worker must arrive in that worker's order; events from different workers may arrive in any order. Unknown events leave the state unchanged apart from the clock.

counted-interrupted? clj

Reasons a namespace was stopped part way that make the run fail. Fail-fast is the parent's own choice, and the engine ignores what such a worker sends afterwards; :stopped comes from a worker that stopped before every var ran without the parent asking, so its unrun vars must not pass silently.

counted-not-run? clj

Reasons a planned namespace did not run that make the run fail. Fail-fast leftovers do not count: Kaocha stops the same way and counts only what ran.

crash-class clj

Error class of the synthetic :ns/error the scheduler emits when a worker dies holding a namespace.

exit-code clj

(exit-code state)

0 when nothing failed, otherwise the number of failures capped at 255, like Kaocha (kaocha.runner/run: (min (+ error fail) 255)). Load errors, unreadable files, crashed workers, protocol lines that could not be read, namespaces that could not be run, planned namespaces that never finished and namespaces a worker stopped on its own (counted-interrupted?) each count as one; assertions whose details were lost count as many as were lost.

failures clj

(failures state)

Every failed assertion, errored assertion, namespace load error, worker crash, hook that failed as a worker shut down (:worker-error), unreadable test file (:file-error), protocol line the parent could not read (:protocol-error) and namespace that could not be run (:not-run, except after fail-fast), ordered by namespace, var, line. Each carries :kind, :ns, and for assertions :var, :line, :message, :contexts, :expected, :actual, :error and the var's captured :output, with :output-at, how much of it was printed before the failure, when the worker captured it.

in-run? clj

(in-run? v)

True for a var that is part of the run: it ran, is running, crashed, or is pending. A var the filters, a pre-test hook or a failed fixture kept from running is not, as Kaocha's kaocha.testable/run-testable gives a skipped testable no count; only pending vars count as skipped.

init clj

(init plan-summary)

Returns an empty run state. plan-summary is {:namespaces [ns-sym ...] :tests {ns-sym n} :started-at millis}; every key is optional. Planned namespaces that have not started count as pending. :tests is the plan's estimate of how many tests each namespace will run, used until its worker counts them (:ns/tests).

interrupted clj

(interrupted state)

Namespaces stopped part way, {ns reason}: with fail-fast, the ones other workers were running when the first failure arrived. They are listed, and count as failures only with reason :stopped (exit-code).

not-run clj

(not-run state)

Planned namespaces that never ran or never finished, {ns reason}. Reason :unfinished marks a namespace that is neither finished nor accounted for otherwise; during a run that is every namespace still waiting or running.

ns-counts clj

(ns-counts {:keys [pass assertions]})

Assertions a namespace made outside its vars (in :once fixtures): {:pass n :fail n :error n}.

ns-failures-of clj

(ns-failures-of state ns)

failures of namespace ns alone, in the same order: what (filter #(= ns (:ns %)) (failures state)) gives, without looking at every other namespace.

ns-result clj

(ns-result {:keys [status vars protocol-error? assertions]})

One of :running, :passed, :failed, :error, :crashed, :interrupted.

result clj

(result state)

Plain data about the finished run, for --print-result and for comparing against Kaocha: per-var status and counts keyed by fully qualified symbol, per-namespace result, the totals Kaocha's summary line shows (:tests, and :pending, which is what volley's summary calls skipped), unreadable files, namespaces that never ran, and the exit code.

run-vars clj

(run-vars rec)

The vars of namespace record rec that are part of the run (in-run?).

running clj

(running state)

What each busy worker is doing: {:worker :ns :var :ns-at :var-at :vars-done} ordered by worker id. :var and :var-at are nil between tests.

slowest clj

(slowest state n)

The n slowest vars as {:ns :var :duration-ms}, slowest first.

summary clj

(summary state)

(summary state now)

Counts for the run so far. now defaults to the latest event time. [:vars :total] counts the tests seen so far, [:vars :expected] the tests the run is expected to have once every namespace has run, and [:vars :estimated?] is true while that number rests on the plan's estimates.

unselected? clj

(unselected? {:keys [status vars protocol-error? assertions pass]})

True for a namespace that finished without running or reporting anything because the filters selected none of its vars (or it has none). The plan cannot always tell beforehand, since tests defined by macros are invisible to it. Kaocha does not report such a namespace (kaocha.testable/run-testable skips a group whose tests are all skipped), so it is neither listed nor counted.

var-counts clj

(var-counts {:keys [pass assertions] :as v})

Assertion counts for a var record: {:pass n :fail n :error n}. Failures and errors are counted from assertion events, or from the counts in :var/end when those are larger (assertion details were lost); passes come from :var/end.

var-output clj

(var-output {:keys [stray output]})

Everything a var printed: stray lines first, then captured output.

var-status clj

(var-status {:keys [phase] :as v})

One of :running, :passed, :failed, :errored, :skipped, :crashed.