Reporters generate textual output during a test run, providing real-time
information on test progress, failures, errors, and so forth. They are in
nature imperative and side-effectful, they generate output on an output
stream (typically stdout), based on test events. Some reporters are also used
to track state. This is unfortunate as it goes against Kaocha's functional
design, but since we want test runs to be interruptible it is somewhat
The concept of reporters is directly taken from clojure.test, but is used in
Kaocha also when running other types of tests.
A reporter is a function which takes a single argument, a map. The map will
:type key indicating the type of event, e.g.
Reporters as imagined in
clojure.test are a flawed design, we try to make
the best of it. See also the monkeypatching of
kaocha.monkey-patch, which is necessary to be able to intercept failures
quickly in case the users runs with
--fail-fast enabled. The patch also
ensures that the current testable is always available in the event map under
Kaocha differs from stock
clojure.test in that multiple reporters can be
active at the same time. On the command line you can specify
multiple times, in
tests.edn you can pass a vector to
and/or point at a var which itself defines a vector of functions. Each of the
given functions will be called in turn for each event generated.
This has allowed Kaocha to split the functionality of reporters up, making
them more modular. E.g.
kaocha.report/report-counters only keeps the
fail/error/pass/test counters, without concerning itself with output, making
This namespace implements the reporters provided by Kaocha out of the box that
don't need extra dependencies. Others like e.g. the progress bar are in their
own namespace to prevent loading files we don't need, and thus slowing down
clojure.test provides reporters as a way to extend the library. By default
clojure.test/report is a multimethod which dispatches on
:type, and so
libraries can extend this multimethod to add support for their own event
types. A good example is the
:mismatch event generated by
Tools can also rebind
clojure.test/report, and use it as an interface for
capturing test run information.
The problem is that these two approaches don't mesh. When tools (like Kaocha,
CIDER, Cursive, etc.) rebind
clojure.test/report, then any custom extensions
to the multimethod disappear.
This can also cause troubles when a library which extends
clojure.test/report gets loaded after it has been rebound. This was an issue
for a while in test.check, which assumed
report would always be a
multimethod (this has been rectified). For this reasons Kaocha only rebinds
report after the "load" step.
Kaocha tries to work around these issues to some extent by forwarding any keys
it does not know about to the original
clojure.test/report multimethod. This
isn't ideal, as these extensions are not aware of Kaocha's formatting and
output handling, but it does provide some level of compatiblity with third
For popular libraries we will include reporter implementations that handle
these events in a way that makes sense within Kaocha, see e.g.
kaocha.matcher-combinators. Alternatively library authors can
themselves strive for Kaocha compatiblity, we try to give them the tools to
enable this, through keyword derivation and custom multimethods.
kaocha.report makes use of Clojure's keyword hierarchy feature to determine
the type of test events. To make Kaocha aware of your custom event, first add
a derivation from
:kaocha/known-type, this will stop the event from being
propagated to the original
(kaocha.hierarchy/derive! :mismatch :kaocha/known-type)
If the event signals an error or failure which causes the test to fail, then
:kaocha/fail-type. This will make Kaocha's existing reporters
compatible with your custom event.
(kaocha.hierarchy/derive! :mismatch :kaocha/fail-type)