Golden-trace testing: assert that a run's execution matches a recorded baseline, not just its final output.
Trace some code, run it, and compare the recorded call tree - functions called, arguments, return values, nesting, and (with inner tracing) the intermediate expression values - against a stored golden file. When the trace changes, the test fails with a diff; when the change is intentional, regenerate the golden.
This is regression testing at the level of how your code ran, which a plain
return-value assertion can't see and neither tools.trace nor a stepping
debugger offers.
Typical use in clojure.test:
(require '[sayid.core :as sd] '[sayid.golden :as gold])
(deftest orders-golden
(sd/ws-reset!)
(sd/ws-add-trace-ns! my.orders)
(my.orders/place-order sample-order)
(is (gold/matches-golden? "place-order")))
The first run writes test/golden/place-order.edn (review and commit it);
later runs compare against it. Regenerate goldens by binding *update* true or
setting the SAYID_GOLDEN_UPDATE environment variable.
Golden-trace testing: assert that a run's *execution* matches a recorded
baseline, not just its final output.
Trace some code, run it, and compare the recorded call tree - functions called,
arguments, return values, nesting, and (with inner tracing) the intermediate
expression values - against a stored golden file. When the trace changes, the
test fails with a diff; when the change is intentional, regenerate the golden.
This is regression testing at the level of *how* your code ran, which a plain
return-value assertion can't see and neither `tools.trace` nor a stepping
debugger offers.
Typical use in `clojure.test`:
(require '[sayid.core :as sd] '[sayid.golden :as gold])
(deftest orders-golden
(sd/ws-reset!)
(sd/ws-add-trace-ns! my.orders)
(my.orders/place-order sample-order)
(is (gold/matches-golden? "place-order")))
The first run writes `test/golden/place-order.edn` (review and commit it);
later runs compare against it. Regenerate goldens by binding `*update*` true or
setting the `SAYID_GOLDEN_UPDATE` environment variable.Directory the golden .edn files live in. Defaults to test/golden.
Directory the golden `.edn` files live in. Defaults to `test/golden`.
When true, check-golden (re)writes the golden instead of comparing. Defaults
to whether the SAYID_GOLDEN_UPDATE environment variable is set.
When true, `check-golden` (re)writes the golden instead of comparing. Defaults to whether the `SAYID_GOLDEN_UPDATE` environment variable is set.
(capture-baseline!)Snapshot the active workspace's normalized trace as the baseline that
diff-from-baseline compares against. For the "snapshot, change the code,
compare" workflow. Returns the number of root calls captured.
Snapshot the active workspace's normalized trace as the baseline that `diff-from-baseline` compares against. For the "snapshot, change the code, compare" workflow. Returns the number of root calls captured.
(check-golden name data)Compare DATA against the stored golden NAME. Returns a map whose :status is
one of :created (no golden existed - wrote it), :updated (*update* was on -
rewrote it), :match, or :mismatch (with a :diff of
[in-golden-only in-run-only]).
Compare DATA against the stored golden NAME. Returns a map whose `:status` is one of `:created` (no golden existed - wrote it), `:updated` (`*update*` was on - rewrote it), `:match`, or `:mismatch` (with a `:diff` of [in-golden-only in-run-only]).
(diff-from-baseline)Diff the active workspace's trace against the last capture-baseline!
snapshot, or nil if no baseline has been captured yet.
Diff the active workspace's trace against the last `capture-baseline!` snapshot, or nil if no baseline has been captured yet.
(diff-traces a b)Structurally diff two normalized traces (as returned by golden-trace),
aligning calls by position. Returns a pruned tree of just the differences: each
changed call with the fields that differ ([in-a in-b]) and its changed
children, and calls present in only one run tagged :added / :removed. An empty
result means the two executions ran identically - which is a handy thing to
assert around a refactor:
(sd/ws-reset!) (sd/ws-add-trace-ns! my.ns) (my.ns/run)
(def before (gold/golden-trace))
;; ... change the code, reload ...
(sd/ws-reset!) (sd/ws-add-trace-ns! my.ns) (my.ns/run)
(gold/diff-traces before (gold/golden-trace)) ; [] => behaviour unchanged
Structurally diff two normalized traces (as returned by `golden-trace`),
aligning calls by position. Returns a pruned tree of just the differences: each
changed call with the fields that differ (`[in-a in-b]`) and its changed
children, and calls present in only one run tagged :added / :removed. An empty
result means the two executions ran identically - which is a handy thing to
assert around a refactor:
(sd/ws-reset!) (sd/ws-add-trace-ns! my.ns) (my.ns/run)
(def before (gold/golden-trace))
;; ... change the code, reload ...
(sd/ws-reset!) (sd/ws-add-trace-ns! my.ns) (my.ns/run)
(gold/diff-traces before (gold/golden-trace)) ; [] => behaviour unchanged(golden-trace & [w])The normalized golden trace of the active workspace (or workspace W): a vector of the recorded root call trees.
The normalized golden trace of the active workspace (or workspace W): a vector of the recorded root call trees.
(matches-golden? name & [w])Assert the active workspace's normalized trace matches golden NAME. Returns
true on a match (or when the golden was just created/updated, printing a notice);
on a mismatch prints the diff and returns false, so it reads naturally inside
(is (matches-golden? ...)).
Assert the active workspace's normalized trace matches golden NAME. Returns true on a match (or when the golden was just created/updated, printing a notice); on a mismatch prints the diff and returns false, so it reads naturally inside `(is (matches-golden? ...))`.
(normalize-node node)Project a recorded node onto the stable, comparable fields: the function or expression, its arguments and return-or-throw as printed values, inner-trace tags, and children. Drops everything volatile - ids, paths, timings, source metadata, and the throw's stacktrace.
Project a recorded node onto the stable, comparable fields: the function or expression, its arguments and return-or-throw as printed values, inner-trace tags, and children. Drops everything volatile - ids, paths, timings, source metadata, and the throw's stacktrace.
(read-golden name)Read the stored golden NAME, or nil if it doesn't exist yet.
Read the stored golden NAME, or nil if it doesn't exist yet.
(write-golden! name data)Write DATA as the golden NAME, creating the directory if needed. Returns the file.
Write DATA as the golden NAME, creating the directory if needed. Returns the file.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |