stateful-check.core — org.clojars.czan/stateful-check 0.4.4

*run-commands* clj

build-test-runner clj

(build-test-runner specification commands timeout-ms assume-immutable-results)

combine-cmds-with-traces clj

(combine-cmds-with-traces command result result-str)

default-assume-immutable-results clj

default-max-tries clj

default-num-tests clj

default-timeout-ms clj

print-execution clj

(print-execution {:keys [message sequential parallel failures]} stacktrace?)

report-result clj

(report-result msg _ options results frequencies)

run-specification clj

(run-specification specification)

(run-specification specification options)

Run a specification. This will convert the spec into a property and run it using clojure.test.check/quick-check. This function then returns the full quick-check result.

spec->property clj

(spec->property spec)

(spec->property spec options)

Turn a specification into a testable property.

specification-correct? clj

(specification-correct? specification)

(specification-correct?
  specification
  {:report {:command-frequency? false :first-case? false :stacktrace? false}
   :run {:max-tries 1
         :num-tests 200
         :seed (java.lang.System/currentTimeMillis)
         :timeout-ms 0}
   :gen {:max-length 5
         :max-size 200
         :shrink-strategies stateful-check.generator/default-shrink-strategies
         :threads 0}})

Test whether or not the specification matches reality. This generates test cases and runs them. If run with in an is, it will report details (and pretty-print them) if it fails.

The options map consists of three potential keys: :gen, :run, and :report, each of which influence a different part of the test.

:gen has four sub-keys:

• :threads specifies how many parallel threads to execute
• :max-length specifies a max length for command sequences
• :max-size specifies a maximum size for generated values
• :shrink-strategies specifies a sequence of shrink strategies that should be tried (in order) to reduce the size of a failing test (see stateful-check.generator/default-shrink-strategies and stateful-check.shrink-strategies)

:run has three sub-keys:

• :max-tries specifies how attempts to make to fail a test
• :num-tests specifies how many tests to run
• :seed specifies the initial seed to use for generation
• :timeout-ms specifies the maximum number of milliseconds that a test is permitted to run for - taking longer is considered a failure (default: 0, meaning no timeout; see NOTE below for more details)
• :assume-immutable-results specifies whether the runner should assume that the results of running commands are immutable, and thus delay string converstions until the end of the test run (default: false)

:report has two sub-keys, but only works within an is:

• :first-case? specifies whether to print the first failure
• :stacktrace? specifies whether to print exception stacktraces
• :command-frequency? specifies whether to print information about how often each command was run

The :timeout-ms option is unsafe in general, but may be helpful in some circumstances. It allows you to categorise a test as a failure if it takes more than a given time, but each of the threads must respond to being interrupted by completing and shutting down. If these threads do not shut themselves down then they may continue to consume system resources (CPU and memory, among other things), impacting other tests.