Test Runners

As you have learned, the poly tool can run your tests incrementally. It creates an isolated classloader for each project and runs your tests within that classloader. An in-process runner avoids process startup time costs. It runs all tests in the same process.

For in-process test runners, the poly tool determines which bricks and projects are affected, calculates the classpath for each project, creates an isolated classloader, and passes it to the test runner for execution.

The poly tool, by default, uses an in-process test-runner.

The alternative to the in-process isolated classloader approach is to execute each project’s tests in a separate external Java subprocess. The advantage of this approach is that you have complete control of the test execution, and memory is released after each project when the subprocess terminates. Use an external test runner if you have issues with the in-process test-runner.

The poly tool will still determine the changed bricks and projects and calculate the classpath for each project. However, the external test runner will launch a subprocess to run the tests.

To use a custom test runner, add it to your poly tool classpath. An ideal place to do this is the :poly alias in your ./deps.edn file:

Once you have your test runner on the classpath, add it to our workspace configuration so that the poly tool can use it instead of the default test runner.

You can override the default global test runner under the :test key in your workspace.edn. Here are three examples:

You can override the global test runner for a project. Add a :test key in the project’s configuration in your workspace.edn to select project-specific test runners:

Polylith supports writing test runners with two protocols: TestRunner and ExternalTestRunner. You can use these protocols to easily plugin custom test runner(s) into the poly tool.

See our relevant API docs:

To implement your custom test runner, create a single-arity constructor function that reifies the TestRunner protocol. Optionally, reify the ExternalTestRunner protocol to make an external test runner.

The poly tool will call your constructor function to get an instance of your test runner for each project test run. The constructor function will receive a map as the single argument:

If you create your own test runner or use one from the community, it can be useful to add snippets of test configuration when running the test command.

The test snippets can be configured in workspace.edn under :test-configs (see the test-runners example):

If we run the test command, e.g. poly test, then the test configuration specified by the :test key will be passed to the test runner(s). Let’s start a shell and verify this, from the examples/test-runners directory:

Now you can check the content of the :test key:

Yes, it looked the same. Let’s use the default test runner:

The old org.corfield.external-test-runner.interface/create value was replaced by [:default].

If the old value instead was set to [org.corfield.external-test-runner.interface/create] then the two vectors would instead been merged into [org.corfield.external-test-runner.interface/create :defalt].

Both default-test-runner and kaocha-test-runner store their values in a vector, and if we select both, the result will be merged:

Here is another example, where :integration and :dummy are concatenated:

Now when we understand how test configuration can be added, we can run the test command using the default test runner:

The default test runner works fine in most cases and is simple and fast. Sometimes, using the same classloader for all your tests in the workspace doesn’t give enough isolation. In this case, the External Test Runner is a good choice. If you switch to the Kaocha Test Runner, you will get more options for running your tests.