kaocha.report
Reporters generate textual output during a test run, providing real-time information on test progress, failures, errors, and so forth. They are in nature imperative and side-effectful, they generate output on an output stream (typically stdout), based on test events. Some reporters are also used to track state. This is unfortunate as it goes against Kaocha's functional design, but since we want test runs to be interruptible it is somewhat inevitable.
The concept of reporters is directly taken from clojure.test, but is used in Kaocha also when running other types of tests.
A reporter is a function which takes a single argument, a map. The map will
have a :type key indicating the type of event, e.g. :begin-test-var,
:fail, :pass, or :summary.
Reporters as imagined in clojure.test are a flawed design, we try to make
the best of it. See also the monkeypatching of clojure.test/do-test in
kaocha.monkey-patch, which is necessary to be able to intercept failures
quickly in case the users runs with --fail-fast enabled. The patch also
ensures that the current testable is always available in the event map under
:kaocha/testable,
Kaocha differs from stock clojure.test in that multiple reporters can be
active at the same time. On the command line you can specify --reporter
multiple times, in tests.edn you can pass a vector to :kaocha/reporter,
and/or point at a var which itself defines a vector of functions. Each of the
given functions will be called in turn for each event generated.
This has allowed Kaocha to split the functionality of reporters up, making
them more modular. E.g. kaocha.report/report-counters only keeps the
fail/error/pass/test counters, without concerning itself with output, making
it reusable.
This namespace implements the reporters provided by Kaocha out of the box that don't need extra dependencies. Others like e.g. the progress bar are in their own namespace to prevent loading files we don't need, and thus slowing down startup.
Issues with clojure.test reporters
clojure.test provides reporters as a way to extend the library. By default
clojure.test/report is a multimethod which dispatches on :type, and so
libraries can extend this multimethod to add support for their own event
types. A good example is the :mismatch event generated by
matcher-combinators.
Tools can also rebind clojure.test/report, and use it as an interface for
capturing test run information.
The problem is that these two approaches don't mesh. When tools (like Kaocha,
CIDER, Cursive, etc.) rebind clojure.test/report, then any custom extensions
to the multimethod disappear.
This can also cause troubles when a library which extends
clojure.test/report gets loaded after it has been rebound. This was an issue
for a while in test.check, which assumed report would always be a
multimethod (this has been rectified). For this reasons Kaocha only rebinds
report after the "load" step.
Kaocha tries to work around these issues to some extent by forwarding any keys
it does not know about to the original clojure.test/report multimethod. This
isn't ideal, as these extensions are not aware of Kaocha's formatting and
output handling, but it does provide some level of compatiblity with third
party libraries.
For popular libraries we will include reporter implementations that handle
these events in a way that makes sense within Kaocha, see e.g.
kaocha.matcher-combinators. Alternatively library authors can
themselves strive for Kaocha compatiblity, we try to give them the tools to
enable this, through keyword derivation and custom multimethods.
Custom event types
kaocha.report makes use of Clojure's keyword hierarchy feature to determine
the type of test events. To make Kaocha aware of your custom event, first add
a derivation from :kaocha/known-type, this will stop the event from being
propagated to the original clojure.test/report
(kaocha.hierarchy/derive! :mismatch :kaocha/known-key)
If the event signals an error or failure which causes the test to fail, then
derive from :kaocha/fail-type. This will make Kaocha's existing reporters
compatible with your custom event.
(kaocha.hierarchy/derive! :mismatch :kaocha/fail-type)
Reporters generate textual output during a test run, providing real-time information on test progress, failures, errors, and so forth. They are in nature imperative and side-effectful, they generate output on an output stream (typically stdout), based on test events. Some reporters are also used to track state. This is unfortunate as it goes against Kaocha's functional design, but since we want test runs to be interruptible it is somewhat inevitable. The concept of reporters is directly taken from clojure.test, but is used in Kaocha also when running other types of tests. A reporter is a function which takes a single argument, a map. The map will have a `:type` key indicating the type of event, e.g. `:begin-test-var`, `:fail`, `:pass`, or `:summary`. Reporters as imagined in `clojure.test` are a flawed design, we try to make the best of it. See also the monkeypatching of `clojure.test/do-test` in `kaocha.monkey-patch`, which is necessary to be able to intercept failures quickly in case the users runs with `--fail-fast` enabled. The patch also ensures that the current testable is always available in the event map under `:kaocha/testable`, Kaocha differs from stock `clojure.test` in that multiple reporters can be active at the same time. On the command line you can specify `--reporter` multiple times, in `tests.edn` you can pass a vector to `:kaocha/reporter`, and/or point at a var which itself defines a vector of functions. Each of the given functions will be called in turn for each event generated. This has allowed Kaocha to split the functionality of reporters up, making them more modular. E.g. `kaocha.report/report-counters` only keeps the fail/error/pass/test counters, without concerning itself with output, making it reusable. This namespace implements the reporters provided by Kaocha out of the box that don't need extra dependencies. Others like e.g. the progress bar are in their own namespace to prevent loading files we don't need, and thus slowing down startup. ### Issues with clojure.test reporters `clojure.test` provides reporters as a way to extend the library. By default `clojure.test/report` is a multimethod which dispatches on `:type`, and so libraries can extend this multimethod to add support for their own event types. A good example is the `:mismatch` event generated by matcher-combinators. Tools can also rebind `clojure.test/report`, and use it as an interface for capturing test run information. The problem is that these two approaches don't mesh. When tools (like Kaocha, CIDER, Cursive, etc.) rebind `clojure.test/report`, then any custom extensions to the multimethod disappear. This can also cause troubles when a library which extends `clojure.test/report` gets loaded after it has been rebound. This was an issue for a while in test.check, which assumed `report` would always be a multimethod (this has been rectified). For this reasons Kaocha only rebinds `report` *after* the "load" step. Kaocha tries to work around these issues to some extent by forwarding any keys it does not know about to the original `clojure.test/report` multimethod. This isn't ideal, as these extensions are not aware of Kaocha's formatting and output handling, but it does provide some level of compatiblity with third party libraries. For popular libraries we will include reporter implementations that handle these events in a way that makes sense within Kaocha, see e.g. `kaocha.matcher-combinators`. Alternatively library authors can themselves strive for Kaocha compatiblity, we try to give them the tools to enable this, through keyword derivation and custom multimethods. ### Custom event types `kaocha.report` makes use of Clojure's keyword hierarchy feature to determine the type of test events. To make Kaocha aware of your custom event, first add a derivation from `:kaocha/known-type`, this will stop the event from being propagated to the original `clojure.test/report` ``` clojure (kaocha.hierarchy/derive! :mismatch :kaocha/known-key) ``` If the event signals an error or failure which causes the test to fail, then derive from `:kaocha/fail-type`. This will make Kaocha's existing reporters compatible with your custom event. ``` clojure (kaocha.hierarchy/derive! :mismatch :kaocha/fail-type) ```
assertion-typeclj
(assertion-type m)Given a clojure.test event, return the first symbol in the expression inside (is).
Given a clojure.test event, return the first symbol in the expression inside (is).
dispatch-extra-keysclj
(dispatch-extra-keys m)Call the original clojure.test/report multimethod when dispatching an unknown key. This is to support libraries like nubank/matcher-combinators that extend clojure.test/assert-expr, as well as clojure.test/report, to signal special conditions.
Call the original clojure.test/report multimethod when dispatching an unknown key. This is to support libraries like nubank/matcher-combinators that extend clojure.test/assert-expr, as well as clojure.test/report, to signal special conditions.
dotsclj
Reporter that prints progress as a sequence of dots and letters.
Reporter that prints progress as a sequence of dots and letters.
fail-fastclj
(fail-fast m)Fail fast reporter, add this as a final reporter to interrupt testing as soon as a failure or error is encountered.
Fail fast reporter, add this as a final reporter to interrupt testing as soon as a failure or error is encountered.
testing-vars-strclj
(testing-vars-str {:keys [file line testing-vars kaocha/testable] :as m})Returns a string representation of the current test. Renders names in :testing-vars as a list, then the source file and line of current assertion.
Returns a string representation of the current test. Renders names in :testing-vars as a list, then the source file and line of current assertion.
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close