User Guide — com.github.lread/test-doc-blocks 1.0.146-alpha

User Guide

Audience

You are someone who wants to verify that the Clojure/ClojureScript code examples in your docs work as expected.

Prerequisites

Test generation requires Clojure v1.9 or above.

Generated tests can be run on Clojure v1.8 and above (if supported by your test runner).

You’ll use your test runner of choice, for example: Clojure test-runner, cljs-test-runner or kaocha.

Introduction

Test-doc-blocks might (or might not, see limitations, and interesting alternatives) be of interest for your Clojure/ClojureScript project.

Test-doc-blocks recognizes:

CommonMark code blocks found in docstrings, .md files and .adoc files.
AsciiDoc code blocks found in .adoc files.

An AsciiDoc code block looks like this:

[source,clojure]
----
;; Some valid Clojure code here.
----

A CommonMark (sometimes also referred to as GitHub Markdown) code block looks like this:

```Clojure
;; Some valid Clojure code here.
```

If the code in your code blocks are REPL style:

user=> (* 6 7)
42

Or editor style:

(* 6 7)
;; => 42

And don’t rely on any special unspecified setup, then test-doc-blocks will generate tests that verify that, yes, indeed:

(* 6 7) evaluates to 42.

Usage

Add an alias to your deps.edn:

    :gen-doc-tests {:extra-deps {com.github.lread/test-doc-blocks {:mvn/version "1.0.146-alpha"}}
                    :exec-fn lread.test-doc-blocks/gen-tests}

The most basic usage is:

clojure -X:gen-doc-tests

This generates Clojure tests for code blocks in your README.md file to the target/test-doc-blocks/test directory. Any existing tests under target/test-doc-blocks will be replaced.

The generated tests have no dependency on test-doc-blocks. You can run them with your preferred test runner.

Test-doc-blocks looks for assertions in your doc code blocks that are in editor style and REPL style format. For example…

user=> (/ 714 17)
42

…generates the test assertion (is (= 42 (/ 714 17))).

Detailed Examples with Inline Options

For detailed examples and inline options:

deps.edn Examples

The test-doc-blocks deps.edn has some aliases that might serve as useful examples:

:isolated/cljs-test-runner - runs generated tests under ClojureScript using cljs-test-runner
Invoke for this project via: clj -M:isolated/cljs-test-runner
:isolated/kaocha - runs generated tests under Clojure using kaocha
Invoke for this project via: clj -M:isolated/kaoacha generated. Note also kaocha tests.edn config which tells koacha to run tests in order (rather than randomized order).
:isolated/clj-test-runner - runs generated tests under Clojure using Cognitect test-runner
Invoke for this project via: clj -M:isolated/clj-test-runner

Command Line Options

:docs

By default, tests are generated for README.md only.

If you want to specify a different vector of files you can do so via :docs:

clojure -X:gen-doc-tests :docs '["README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]'

:docs understands glob syntax.

:target-root

The default directory where tests are generated is ./target.

You can override this via :target-root:

clojure -X:gen-doc-tests :target-root '"./someplace/else"'

Note that test-doc-blocks will delete and recreate the test-docs-block/test dir under the target root. Keep the target location in mind when figuring out where to point your test runner.

:platform

The platform governs what Clojure file types test-doc-blocks generates for tests.

Specify:

:clj for Clojure, generates .clj files
:cljs for ClojureScript, generates .cljs files
:cljc for mixed, generates .cljc files

The default is :cljc. You can override the platform for code blocks via inline options.
Test-doc-blocks makes no platform assumptions when generating tests from doc blocks found in Clojure source files. Specify what makes sense for your tests.

Windows users: trying to escape -X args on the command line can lead to frustration. Consider instead specifying options under :exec-args in your :gen-doc-tests alias in deps.edn.

Limitations

Some limitations that we might entertain addressing:

If your code block depends on some external setup, we’ve no way to express that.
Test-doc-blocks will automatically handle inline (require …) and (import …) appearing in code blocks, but not in any complex expressions of these forms.
Parsing adoc and md files is on the naive side but should handle most common cases. If we’ve overlooked a common syntax, let us know.

Some limitations we have no current plans to address:

Code blocks using ns or in-ns will not work with test-doc-blocks.
For REPL style code blocks, we only look for user⇒ prompts and no other ns prompts.
It is possible to embed HTML into your docs. If you express code or headings in embedded HTML within your doc, test-doc-blocks won’t find them.

Interesting Alternatives

Other options and related projects that I am currently aware of:

kaocha - Kaocha supports running cucumber tests. It uses this support in tests for some of its documentation. A .feature document describes the feature and includes given, when, then scenarios that are both run and shown in the documentation. You can use step definitions to hide any gritty details.
readme - Generates tests for code blocks found in .md files and then runs them.
testdoc - Tests code blocks in docstrings and external docs.
alc.x-as-tests - Runs code in (comment …) blocks as tests.
notebook - Some day notebook type tools might serve both as tests and docs