Polylith CI setup — polylith/clj-poly 0.2.20

The Polylith CI setup

The polylith repository has a continuous integration and deployment setup via CircleCI. The setup has a workflow for validation, testing, and deployment. The workflow has several jobs that validate, test, and deploy the artifacts to Clojars, cljdoc, and GitHub as releases.

You can find the CircleCI configuration file here.

Releases

A release is triggered when a commit is pushed to master, often a squashed branch that fixes/implements an issue. If SNAPSHOT is included in the version name, then it becomes a snapshot release, otherwise a "final" release.

Final release

A commit is pushed to the master branch where version is not a snapshot, e.g. 0.2.18 or 0.2.19, which triggers a build in CircleCi that:

Creates a release of the poly tool.
Releases the clj-poly library to Clojars.
Triggers a build of the web based cljdoc documentation.

Snapshot release

A commit is pushed to the master branch where version is a snapshot, e.g. 0.2.19-SNAPSHOT, which triggers a build in CircleCi that:

Creates a pre-release of the poly tool.
Releases the snapshot version of the clj-poly library to Clojars.
Triggers a snapshot build of the web based cljdoc documentation.

The same snapshot release can be triggered several times and will overwrite the latest snapshot release.

The next-release.adoc file lists what has been included in the SNAPSHOT release, since the previous final release. This file is cleared when making the first SNAPSHOT release directly after a final release. Changes to the workspace structure are listed in the versions page.

Old SNAPSHOT releases are regularly deleted. If you use snapshot releases, ensure you use the latest SNAPSHOT or stable version.

Deploy Workflow

This workflow has three jobs triggered by any commit to a branch associated with a pull request. These three jobs are check, info, and test. The workflow runs the remaining jobs only if the current branch is the master branch.

Jobs

create-version-txt

This job runs the create-version-txt function in the build.clj file. The create-version-txt function gets the current version string and writes it to the version.txt file. This file will be persisted during the CI run so that the following jobs can read the version from it.

check

This job runs the check command from the poly tool as follows: clojure -M:poly check. If there are any errors in the workspace, it returns with a non-zero exit code and the CircleCI workflow stops at this stage. If there are any warnings printed by the tool, it will be visible in the job’s output. If the current branch is the master branch, since:previous-release argument is added to the command and run as: clojure -M:poly check since:previous-release.

info

This job runs the following commands, one after another:

clojure -M:poly ws
- Prints the current workspace as data in edn format.
clojure -M:poly info
- Prints workspace information.
clojure -M:poly deps
- Prints the dependency information
clojure -M:poly libs
- Prints all libraries that are used in the workspace.

If the current branch is the master branch, since:previous-release argument is added to each command above.

After this job is done, all this information will be available in the jobs output for debugging purposes, if needed. You can read more about available commands here.

test

This job runs all the tests for all the bricks and projects that are directly or indirectly changed since the last stable point in time. The poly tool supports incremental testing out-of-the-box by using stable point marks in the git repository. It runs the following command: clojure -M:poly test :project. If any of the tests fail, it will exit with a non-zero exit code and the CircleCI workflow stops at this stage. Information about the passed/failed tests will be printed in the job’s output. If the current branch is the master branch, since:previous-release argument is added to the command and run as: clojure -M:poly test since:previous-release.

mark-as-stable

This job only runs for the commits made to the master branch. It adds (or replaces if already exists) the stable-master tag to the repository. At this point in the workflow, it is "proven" that the Polylith workspace is valid and all of the tests are passed, and it’s safe to mark this commit as stable.

It does that by running the following commands, one after another:

git tag -f -a "stable-$CIRCLE_BRANCH" -m "[skip ci] Added Stable Polylith tag"
- Creates or moves the tag.
git push origin $CIRCLE_BRANCH --tags --force-with-lease=$CIRCLE_BRANCH
- Pushes the tag back to the git repository.

add-version-tag

This job only runs for the commits made to the master branch. It adds a new version tag to the repository in the form of vX.Y.Z. The version string is read from the version.txt file. This job skips adding the version tag if the current version is a snapshot version.

The tag is added by running the following commands, one after another:

VERSION=`cat version.txt`
- Reads the version string.
git tag -f -a "v$VERSION" -m "[skip ci] Added new version tag"
- Creates the tag.
git push origin $CIRCLE_BRANCH --tags --force-with-lease=$CIRCLE_BRANCH
- Pushes the tag back to the git repository.

deploy

This job only runs for the commits made to the master branch. It deploys the changed projects to Clojars. If this is a snapshot version, it also triggers a cljdoc build after deploying projects to Clojars. It’s easy to deploy incrementally with the poly tool. Changed projects are calculated since the latest release. You can see how it’s done here. In a nutshell, it executes poly ws get:changes:changed-or-affected-projects skip:dev since:previous-release and only deploys the returned projects.

create-artifacts

This job only runs for the commits made to the master branch. It creates two types of artifacts per changed project, an AOT compiled uberjar and a package that can be used to deploy Homebrew. Created artifacts can be found in the artifacts section of this job’s output.

publish-gitHub-release

This job only runs for the commits made to the master branch. It uploads the artifacts created after the previous job and uploads them to a new release in GitHub. If this is a snapshot version, the release in GitHub is marked as pre-release. If there is an existing pre-release in GitHub for the same version and tag, the release is overriden. It makes use of the GHR tool in order to create a new release on GitHub and upload the artifacts.