Spec generators can fail for unexpected reasons. It requires expertise to see where a complex spec fails to generate.
A practical way to work around the problem is to build and test specs incrementally; the clojure.spec design encourages building nested specs from components; speccing collections independently of their components, building keysets (maps) out of individually specced keys allows testing spec compoments independently. It still takes dicipline to test incrementally and backtracing where a definition went wrong can take much longer than is desirable.
Test component specs automatically using proof-specs
. This adds the
requirement that all specs registered in a particular set of
namespaces must have a working generator. proof-spes
will attempt to
generate data for each spec in the selected namespaces and reports on
every failure.
Specs that are only used as return specs for functions must have a generator, even though they won't be used in any other test.
proof-specs
will try to figure out the reason for a spec/generator
not working. For now it will just list all failing generators and the
given errors.
(require '[nl.jomco.proof-specs :refer [proof-specs]])
(proof-specs
:num-vals 10
:include-regexps [#"nl.jomco.*"])
Produces something like:
{:problems #:nl.jomco.blerk{:yelp #error {
:cause "Couldn't satisfy such-that predicate after 100 tries."
:data {...}
:via
[{:type clojure.lang.ExceptionInfo
:message "Couldn't satisfy such-that predicate after 100 tries."
:data {...}
:at ...}]
:trace
[...]}},
:specs
#{:nl.jomco.blerk/yelp
:nl.jomco.proof-specs/error
:nl.jomco.proof-specs/exclude-regexps
:nl.jomco.proof-specs/include-regexps
:nl.jomco.proof-specs/num-vals
:nl.jomco.proof-specs/problems
:nl.jomco.proof-specs/proof-opts
:nl.jomco.proof-specs/regexp
:nl.jomco.proof-specs/require-namespaces
:nl.jomco.proof-specs/selector
:nl.jomco.proof-specs/spec
:nl.jomco.proof-specs/specs}}
The return value will not contain a :problems
key when no problems
were found.
Add a "proof-specs" alias to your project.clj
- also make sure you
include proof-specs
as a development dependency.
...
profiles {:dev {:dependencies [[nl.jomco/proof-specs "<VERSION>"]]}}
...
:aliases {"proof-specs" ["run" "-m" "nl.jomco.proof-specs"
"--include-regexps" ".*jomco.*"
"--require-namespaces" "nl.jomco.proof-spec-gen,nl.jomco.blerk"]}
...
And run using:
lein proof-specs
Produces something like:
Problems generating data for 1 out of 12 specs:
:nl.jomco.blerk/yelp
#:nl.jomco.blerk{:yelp #error {
:cause "Couldn't satisfy such-that predicate after 100 tries."
:data {...}, :max-tries 100}
:via
[{:type clojure.lang.ExceptionInfo
:message "Couldn't satisfy such-that predicate after 100 tries."
:data {...}, :max-tries 100}
:at [...]}]
:trace
[...]}}
The exit status will be 0
when no problems were found, otherwise
1
.
Can you improve this documentation? These fine people already did:
Joost Diepenmaat & Remco van 't VeerEdit on sourcehut
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close