Proof Specs

RATIONALE

Spec generators can fail for unexpected reasons. It requires expertise to see where a complex spec fails to generate.

A practical way to work around the problem is to build and test specs incrementally; the clojure.spec design encourages building nested specs from components; speccing collections independently of their components, building keysets (maps) out of individually specced keys allows testing spec compoments independently. It still takes dicipline to test incrementally and backtracing where a definition went wrong can take much longer than is desirable.

RESOLUTION

Test component specs automatically using proof-specs. This adds the requirement that all specs registered in a particular set of namespaces must have a working generator. proof-spes will attempt to generate data for each spec in the selected namespaces and reports on every failure.

CONSEQUENSES

Specs that are only used as return specs for functions must have a generator, even though they won't be used in any other test.

NON-GOALS

proof-specs will try to figure out the reason for a spec/generator not working. For now it will just list all failing generators and the given errors.

Running as a library

(require 'nl.jomco.proof-specs)

(nl.jomco.proof-specs/proof-specs
  :num-vals 10
  :include-regexps [#"your.ns.*"])

Running as a leiningen alias

Add a "proof-specs" alias to your project.clj - also make sure you include proof-specs as a development dependency.

  ...
  profiles {:dev {:dependencies [[nl.jomco/proof-specs "<VERSION>"]]}}
  ...
  :aliases {"proof-specs"  ["run" "-m" "nl.jomco.proof-specs" "--include-regexps" ".*jomco.*"
                            "--require-namespaces" "nl.jomco.proof-spec-gen"]}
  ...

And run using

lein proof-specs