User Guide — com.github.lread/test-doc-blocks 1.0.137-alpha

User Guide

Audience

You are someone who wants to verify that the Clojure/ClojureScript code examples in your docs work as expected.

Prerequisites

Test generation requires Clojure v1.9 or above.

Generated tests have been verified to run on Clojure v1.8 and above (if supported by your test runner).

You’ll use your test runner of choice, for example: Clojure test-runner, cljs-test-runner or kaocha.

Introduction

Test-doc-blocks might (or might not, see limitations, and interesting alternatives) be of interest for your Clojure/ClojureScript project.

Test-doc-blocks recognizes:

CommonMark code blocks found in docstrings, .md files and .adoc files.
AsciiDoc code blocks found in .adoc files.

An AsciiDoc code block looks like this:

[source,clojure]
----
;; Some valid Clojure code here.
----

A CommonMark (sometimes also referred to as GitHub Markdown) code block looks like this:

```Clojure
;; Some valid Clojure code here.
```

If the code in your code blocks are REPL style:

user=> (* 6 7)
42

Or editor style:

(* 6 7)
;; => 42

And don’t rely on any special unspecified setup, then test-doc-blocks will generate tests that verify that, yes, indeed:

(* 6 7) evaluates to 42.

Usage

Add an alias to your deps.edn:

    :gen-doc-tests {:extra-deps {com.github.lread/test-doc-blocks {:mvn/version "1.0.137-alpha"}}
                    :exec-fn lread.test-doc-blocks/gen-tests}

The most basic usage is:

clojure -X:gen-doc-tests

This generates Clojure tests for code blocks in your README.md file to the target/test-doc-blocks/test directory. Any existing tests under target/test-doc-blocks will be replaced.

The generated tests have no dependency on test-doc-blocks and can be run with your preferred test runner.

Test-doc-blocks looks for assertions in your doc code blocks that are in editor style and REPL style format. For example…

user=> (/ 714 17)
42

…generates the test assertion (is (= 42 (/ 714 17))). For detailed examples and inline options:

For those writing docs in AsciiDoc, an AsciiDoc example
For CommonMark authors, a CommonMark example
And for those wanting to test code blocks in their docstrings, Clojure source docstring example

The test-doc-blocks deps.edn has some aliases that might serve as useful examples:

:isolated/cljs-test-runner - runs generated tests under ClojureScript using cljs-test-runner
Invoke for this project via: clj -M:isolated/cljs-test-runner
:isolated/kaocha - runs generated tests under Clojure using kaocha
Invoke for this project via: clj -M:isolated/kaoacha generated. Note also kaocha tests.edn config.
:isolated/clj-test-runner - runs generated tests under Clojure using Cognitect test-runner
Invoke for this project via: clj -M:isolated/clj-test-runner

Command Line Options

:docs

By default, tests are generated for README.md only.

If you want to specify a different vector of files you can do so via :docs:

clojure -X:gen-doc-tests :docs '["README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]'

:docs understands glob syntax.

:target-root

The default directory where tests are generated is ./target.

You can override this via :target-root:

clojure -X:gen-doc-tests :target-root '"./someplace/else"'

Note that test-doc-blocks will delete and recreate the test-docs-block/test dir under the target root. Keep the target location mind when figuring out where to point your test runner.

:platform

The platform governs what Clojure file types will be generated for tests.

Specify:

:clj for Clojure, generates .clj files
:cljs for ClojureScript, generates .cljs files
:cljc for mixed, generates .cljc files

The default is :cljc. Platform can be overridden for code blocks via inline options.
No platform assumptions are currently made when generating test from doc blocks found in Clojure source files. Specify what makes sense for your tests.

Windows users: trying to correctly escape -X args on the command line can lead to frustration. Consider instead specifying options under :exec-args in your :gen-doc-tests alias in deps.edn.

Limitations

Some limitations that we might entertain addressing:

If your code block depends on some external setup, we’ve no way to express that.
Test-doc-blocks will automatically handle inline (require …) and (import …) appearing in code blocks, but not in any complex expressions of these forms.
Parsing adoc and md files is on the naive side but should handle most common cases. If we’ve overlooked a common syntax, let us know.

Some limitations we have no current plans to address:

Code blocks using ns or in-ns will not work with test-doc-blocks.
For REPL style code blocks we only look for user⇒ prompts and no other ns prompts.
It is possible to embed HTML into your docs. If your code or headings are expressed in embedded HTML within your doc, test-doc-blocks won’t find them.

Interesting Alternatives

Other options and related projects that I am currently aware of:

kaocha - Kaocha supports running cucumber tests. It uses this support in tests for its own documentation. A .feature document describes the feature and includes given, when, then scenarios that are both run and shown in documentation. Gritty details can be hidden in step definitions.
readme - Generates tests for code blocks found in .md files and then runs them.
testdoc - Tests code blocks in docstrings and external docs.
alc.x-as-tests - Runs code in (comment …) blocks as tests.