Liking cljdoc? Tell your friends :D

User Guide

Table of Contents

Audience
Prerequisites
Introduction
Usage
Limitations
Interesting Alternatives

Audience

You are someone who wants to verify that the Clojure/ClojureScript code examples in your docs work as expected.

Prerequisites

Test generation requires Clojure v1.9 or above.

Generated tests can be run on Clojure v1.8 and above (if supported by your test runner).

You’ll use your test runner of choice, for example: Clojure test-runner, cljs-test-runner, kaocha, or some lein test runner.

Introduction

Test-doc-blocks might (or might not, see limitations, and interesting alternatives) be of interest for your Clojure/ClojureScript project.

Test-doc-blocks recognizes:

CommonMark code blocks found in Clojure sources docstrings, .md files and .adoc files.
AsciiDoc code blocks found in .adoc files.

An AsciiDoc code block looks like this:

[source,clojure]
----
;; Some valid Clojure code here.
----

A CommonMark (sometimes also referred to as GitHub Markdown) code block looks like this:

```Clojure
;; Some valid Clojure code here.
```

If the code in your code blocks are REPL style:

user=> (* 6 7)
42

Or editor style:

(* 6 7)
;; => 42

And don’t rely on any special unspecified setup, then test-doc-blocks will generate tests that verify that, yes, indeed:

(* 6 7) evaluates to 42 via test assertion (is (= 42 (* 6 7))).

Usage

Clojure CLI

An example deps.edn alias:

  :gen-doc-tests {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                                 com.github.lread/test-doc-blocks {:mvn/version "1.0.166-alpha"}}
                  ;; for -X syntax support specify exec-fn
                  :exec-fn lread.test-doc-blocks/gen-tests
                  ;; for -M syntax support specify main-opts
                  :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"]}

We use :replace-deps to keep deps to the minimum while generating our tests.

The most basic usage is:

clojure -X:gen-doc-tests

Or:

clojure -M:gen-doc-tests

This generates Clojure tests for code blocks in your README.md file to the target/test-doc-blocks/test directory. Any existing tests under target/test-doc-blocks will be replaced.

The generated tests have no dependency on test-doc-blocks. You can run them with your preferred test runner.

Windows users: if trying to escape -X args on the command line is leading to frustration, consider:

using the -M syntax
or specifying options under :exec-args or :main-opts in your :gen-doc-tests alias in your deps.edn.

deps.edn Examples

The test-doc-blocks deps.edn has some aliases that might serve as useful examples for running generated tests:

:isolated/cljs-test-runner - runs generated tests under ClojureScript using cljs-test-runner
Invoke for this project via: clj -M:isolated/cljs-test-runner
:isolated/kaocha - runs generated tests under Clojure using kaocha
Invoke for this project via: clj -M:isolated/kaoacha generated.

Review the kaocha tests.edn config which tells koacha to run tests in order (rather than randomized order).
:isolated/clj-test-runner - runs generated tests under Clojure using Cognitect test-runner
Invoke for this project via: clj -M:isolated/clj-test-runner

Leiningen

An an example subset of a project.clj:

(defproject my-lovely-library "0.0.0"
  ;; ...
  :profiles {:gen-doc-tests {:test-paths   ["target/test-doc-blocks/test"]
                             :dependencies [[com.github.lread/test-doc-blocks "1.0.166-alpha"]]}}
  :aliases  {"run-doc-tests" ^{:doc "Generate, then run, tests from doc code blocks"}
             ["with-profile" "gen-doc-tests" "do"
              ["run" "-m" "lread.test-doc-blocks" "gen-tests"
               ;; change gen-tests options as appropriate for your project
               "--platform" "clj"
               "src/**.clj" "doc/example.md"]
              ["test"]]})

Running

lein run-doc-tests

Generates clj tests for all code blocks found in:
- all Clojure source files under src
- doc/example.md
Runs the generated tests.

Detailed Doc Examples with Inline Options

For detailed doc examples that include inline options you will want to read:

Command Line Options

After you experiment with which options are appropriate for your project, you’ll likely incorporate options directly into your deps.edn or project.clj file.

Leiningen users should focus on the -M syntax and apply it back to the leiningen example.

docs

By default, tests are generated for README.md only.

-X syntax
If you want to specify a different vector of files you can do so via :docs

clojure -X:gen-doc-tests :docs '["README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]'

Moving this to a deps.edn alias would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.0.166-alpha"}}
   :exec-fn lread.test-doc-blocks/gen-tests
   :exec-args {:docs ["README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]}}

-M syntax
Equivalent -M syntax is:

clojure -M:gen-doc-tests README.adoc doc/example.adoc doc/example.md doc/example.cljc

Moving this to a deps.edn alias would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.0.166-alpha"}}
   :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"
                    "README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]}

globs
The files you specify can include glob syntax. For example, the following is equivalent to the last example:

clojure -M:gen-doc-tests README.adoc doc/example.{adoc,md,cljc}

When running from a terminal shell, be sure to use any necessary quoting should you want to prevent your shell from interpreting wildcard glob characters, for example, from a bash shell:

clojure -M:gen-doc-tests 'src/**.clj' 'doc/*.md'

Any special quoting should not be necessary when specifying options directly in a deps.edn or project.clj file.

target-root

The default directory where tests are generated is ./target.

-X syntax
Override via :target-root when using the -X syntax:

clojure -X:gen-doc-tests :target-root '"./someplace/else"'

Expressed within a deps.edn alias, this would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.0.146-alpha"}}
   :exec-fn lread.test-doc-blocks/gen-tests
   :exec-args {:target-root "./someplace/else"}}

-M syntax
Use --target-root when using the -M syntax:

clojure -M:gen-doc-tests --target-root ./someplace/else

Expressed within a deps.edn alias, this would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.0.166-alpha"}}
   :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"
                    "--target-root" "./someplace/else"]}

Test-doc-blocks will delete and recreate the test-docs-block/test dir under the target root.

Keep the target location in mind when figuring out where to point your test runner.
If you get the location wrong for your test runner, it will likely not complain; many test runners will happily pass a test run that finds 0 tests.

platform

The platform governs what Clojure file types test-doc-blocks generates for tests.

Specify:

:clj for Clojure, generates .clj files
:cljs for ClojureScript, generates .cljs files
:cljc for mixed, generates .cljc files

The default is :cljc.

-X Syntax Example platform override using -X syntax:

clojure -X:gen-doc-tests :platform :clj

The same, expressed within a deps.edn alias:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.0.166-alpha"}}
   :exec-fn lread.test-doc-blocks/gen-tests
   :exec-args {:platform :clj}}

-M Syntax Same override but using -M syntax:

clojure -M:gen-doc-tests --platform clj

The same, expressed within a deps.edn alias:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.0.166-alpha"}}
   :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"
                    "--platform" "clj"]}

You can override the platform for code blocks via inline options within your docs.

Test-doc-blocks makes no platform assumptions when generating tests from doc blocks found in Clojure source files. Specify what makes sense for your tests.

Limitations

Some limitations that we might entertain addressing:

If your code block depends on some external setup, we’ve no way to express that.
Test-doc-blocks will automatically handle inline (require …) and (import …) appearing in code blocks, but not in any complex expressions of these forms.
Parsing adoc and md files is on the naive side but should handle most common cases. If we’ve overlooked a common syntax, let us know.

Some limitations we have no current plans to address:

Code blocks using ns or in-ns will not work with test-doc-blocks.
For REPL style code blocks, we only look for user⇒ prompts and no other ns prompts.
It is possible to embed HTML into your docs. If you express code or headings in embedded HTML within your doc, test-doc-blocks won’t find them.

Interesting Alternatives

Other options and related projects that I am currently aware of:

readme (archived) - Generates tests for code blocks found in .md files and then runs them. This project was the inspiration for test-doc-blocks. It is simpler but but also has less features.
testdoc - Tests code blocks in docstrings and external docs.
alc.x-as-tests - Runs code in (comment …) blocks as tests.
kaocha - Kaocha supports running Cucumber tests. It uses this support in tests for some of its documentation. A .feature document describes the feature and includes given, when, then scenarios that are both run and shown in the documentation. You can use step definitions to hide any gritty details.
clj-concordian - "A BDD / Specification by Example tool somewhat similar to Cucumber but far simpler", according to its author, go check it out!
notebook - Some day notebook type tools might serve both as tests and docs. Until that golden day, test-doc-blocks and the above tools are worthy of consideration.

CommonMark Example❯

Can you improve this documentation? These fine people already did:
lread, github-actions[bot], Jakub Holý & Lee ReadEdit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

Keyboard shortcuts Report a problem cljdoc on GitHub

× close