Rename-Item $HOME\deps.clj\deps.exe clojure.exe
Rewrite-clj is verified on each push on macOS, Ubuntu and Windows via GitHub Actions.
All scripts are written in Clojure and most invoked via babashka.
This gives us a cross platform scripting language that is familiar, fun and consistent.
These docs will show babashka scripts invoked explicitly via babashka’s bb
; on macOS and linux feel free to leave out bb
.
We make use of planck for cljs bootstrap (aka cljs self-hosted) testing. Planck is currently not available for Windows.
We test that rewrite-clj operates as expected when natively compile via GraalVM. Automated testing is setup using GraalVM v21 for both JDK8 and JDK11. On Windows we only test against JDK11 as tool setup for JDK8 on Windows seemed overly arduous.
Java JDK 1.8 or above
NodeJs v12 or above
Clojure v1.10.1.697 or above for clojure
command
Note that rewrite-clj v1 itself supports Clojure v1.9 and above
Babashka v0.2.3 or above
GraalVM v21.0.0 (if you want to run GraalVM native image tests)
The primary development OSes for rewrite-clj are macOS and Linux. Our line endings are LF only.
I’m not sure what Windows developers typically want for line endings while working on source. I expect, but don’t know, that most Windows editors automatically handle LF as line ending. Someone let me know if I am wrong.
Note that I do explicitly set git’s config core.autocrlf
to false
on our Windows CI unit test environment.
Our import vars code generation checks currently rely on line endings remaining unconverted.
The Clojure story on Windows is still in the early chapters. Scoop offers an easy way to install tools. @littleli is doing a great job w/maintaining scoop apps for Clojure, Babashka and other tools and this is how I installed Babashka.
We all choose our own paths, but for me, using deps.clj instead of Clojure’s PowerShell Module offered me no fuss no muss Clojure on Windows and GitHub Actions on Windows.
I decided to install deps.clj not through scoop but through the deps.clj install.ps1
script.
This makes it simple to treat deps.exe
as if it were the official clojure
via a simple rename:
Rename-Item $HOME\deps.clj\deps.exe clojure.exe
You’ll have your own preference, but I find it convenient to install GraalVM on Windows via scoop.
Both graalvm11
and graalvm8
are available.
You’ll need to load the appropriate Visual C++ environment variables for GraalVM’s native-image to do its work. I found it oddly cumbersome to load them from PowerShell, so I work from a cmd shell instead. Here’s what works on my Windows dev environment:
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat"
And finally, I never did figure out how to get the Windows prerequisites setup for the JDK8 version of GraalVM, so I only test on the JDK11 version.
After checking out this project from GitHub,
Install JavaScript libraries and tools required by doo and shadow-cljs:
sudo npm install karma-cli -g npm install --no-save
The --no-save
tells newer versions of npm
to not automatically upgrade package-lock.json
to the v2 format.
If you are on macOS or linux, install planck.
Initialize cache for clj-kondo so it can lint against your dependencies
bb ./script/lint.clj
Rewrite-clj v0 used a version of potemkin import-vars. Potemkin import-vars copies specified vars from a specified namespace to the current namespace at load time. Due to often mysterious issues related to import-vars, a general dislike for import-vars in the Clojure community, and associated maintenance costs, we’ve opted to instead generate code for rewrite-clj v1.
For any source that used potemkin import-vars, we now have a separate template clj (or cljc) file.
For example src/rewrite_clj/zip.cljc
is generated by template template/rewrite_clj/zip.cljc
.
The syntax of import-vars in the template remains familiar. The following old potemkin import-vars syntax:
(import-vars
[[my.ns1 my-var1 my-var2 my-var3]
[my.ns2 my-var4 my-var5]])
Is expressed in our templates as:
#_{:import-vars/import
{:from [[my.ns1 my-var1 my-var2 my-var3]
[my.ns2 my-var4 my-var5]]}}
We also carry over rewrite-cljc support for :import-vars/import-with-mods
, via an optional :opts
.
See template/rewrite_clj/zip.cljc
for example usage.
Importing will generate delegates.
An import of (defn foo [a b] (+ a b))
from namespace my.ns1
will generate (defn foo [a b] (my.ns1/foo a b))
.
No generation of requires is done, your template will have to require my.ns1
in normal Clojure code.
At this time, we don’t handle destructuring in arglists, and will throw unless args are all symbols.
To generate target source from templates run:
bb script/apply_import_vars.clj gen-code
You are expected to review the generated changes and commit the generated source to version control. We don’t lint templates, but we do lint the generated code.
To perform a read-only check, run:
bb script/apply_import_vars.clj check
The check command will exit with 0 if no changes are required, otherwise it will exit with 1. Our build script will run the check command and fail the build if there are any pending changes that have not been applied.
Your personal preference will likely be different, but during maintenance and refactoring, I found running tests continuously for Clojure and ClojureScript helpful.
For Clojure, I open a shell terminal window and run:
bb ./script/clj_watch.clj
This launches kaocha in watch mode.
For ClojureScript, I open a shell terminal window and run:
bb ./script/cljs_watch.clj
This launches fighweel main. After initialization, your default web browser will automatically be opened with the figwheel auto-testing page.
All documentation is written in AsciiDoc. We follow AsciiDoc best practice of one sentence per line.
Images are created and edited with draw.io desktop. We export to .png with a border of 10 and a transparent background. At the time of this writing draw.io does not remember export settings, so you’ll have to enter them in each time.
We use test-doc-blocks to verify that code blocks in our documentation are in good working order.
bb ./script/doc_tests.clj
This generates tests for doc code blocks and then runs them under Clojure and ClojureScript.
Before pushing, you likely want to mimic what is run on each push via GitHub Actions.
We also verify that rewrite-clj functions as expected when compiled via Graal’s native-image
.
Tests and library natively compiled:
bb ./script/pure_native_test.clj
Library natively compiled and tests interpreted via sci
bb ./script/sci_native_test.clj
To try to ensure our changes to rewrite-clj do not inadvertently break existing popular libraries, we run their tests, or a portion thereof, against rewrite-clj.
bb ./script/libs_tests.clj run
Current libs we test against:
antq
carve
cljfmt
cljstyle
clojure-lsp
mranderson
rewrite-edn
refactor-nrepl
test-doc-blocks
zprint
Additional libs are welcome.
If you are troubleshooting locally, and want to only run specific tests, you can specify which ones you’d like to run. For example:
bb ./script/libs_tests.clj run cljfmt zprint
Running current versions of libs is recommended, but care must be taken when updating. We want to make sure we are patching correctly to use rewrite-clj v1 and running a lib’s tests as intended.
To check for outdated libs:
bb ./script/libs_test.clj outdated
Notes:
libs_tests.clj
was developed on macOS and is run on CI under Linux only under JDK 11 only.
We can expand variations at some later date if there is any value to it.
We test the current HEAD of rewrite-clj v1 against specific versions (latest at the time of this writing) of libs.
We patch lib deps and sometimes code (ex. require
for rewrite-cljc
becomes rewrite-clj
).
As folks migrate to rewrite-clj v1, the need for current patches will lessen.
Updating what versions we test against is currently a manual, but not an overly burdensome, task.
To see what new dependencies are available, run:
bb ./script/outdated.clj
We use antq which also checks pom.xml
.
If you see an outdated dependency reported for pom.xml
after updating deps.edn
, run the following:
clojure -Spom
This script also checks for outdated Node.js dependencies.
Note that checks are only done against installed ./node_modules
, so you may want to run npm install
first.
We use clj-kondo for linting rewrite-clj source code.
We fail the build on any lint violations. The CI server runs:
bb ./script/lint.clj
and you can too.
Integrate clj-kondo into your editor to catch mistakes as they happen.
Rewrite-clj v1’s primary goals include remaining compatible with rewrite-clj v0 and rewrite-cljs and avoiding breaking changes.
To generate reports on differences between rewrite-clj v0, rewrite-cljs and rewrite-clj v1 APIs, run:
bb ./script/gen_api_diffs.clj
Run this script manually on an as-needed basis, and certainly before any official release. Generated reports are to be checked in to version control.
Reports are generated to doc/generated/api-diffs/
and include manually written notes from doc/diff-notes/
.
These reports are referenced from other docs, so if you rename files, be sure to search for links.
Makes use of diff-apis.
Delete .diff-apis/.cache
if you need a clean run.
Before a release, it can be comforting to preview what docs will look like on cljdoc.
Limitations
This script should be considered experimental, I have only tested running on macOS, but am fairly confident it will work on Linux. Not sure about Windows at this time.
You have to push your changes to GitHub to preview them. This allows for a full preview that includes any links (source, images, etc) to GitHub. This works fine from branches and forks - in case you don’t want to affect your main development branch for a preview.
Start Local Services
To start the local cljdoc docker container:
bb ./script/cljdoc_preview.clj start
The local cljdoc server allows your ingested docs to be viewed in your web browser.
The start command also automatically checks docker hub for any updates so that our cljdoc preview matches the current production version of cljdoc.
Ingest Docs
To ingest rewrite-clj API and docs into the local cljdoc database:
bb ./script/cljdoc_preview.clj ingest
The ingest command automatically publishes rewrite-clj to your local maven repository (cljdoc only works with published jars). You’ll have to remember to git commit and git push your changes before ingesting.
Repeat these steps any time you want to preview changes.
Preview Docs
To open a view to the ingested docs in your default web browser:
bb ./script/cljdoc_preview.clj view
If you have just run the start command, be a bit patient, the cljdoc server can take a few moments to start up - especially on macOS due to poor file sharing performance.
Stop Local Services
When you are done, you’ll want to stop your docker container:
bb ./script/cljdoc_preview.clj stop
This will also delete temporary files created to support your preview session, most notably the local cljdoc database.
Note that NO cleanup is done for any rewrite-clj artifacts published to your local maven repository.
Container Status
If you forget where you are at with your docker containers, run:
bb ./script/cljdoc_preview.clj status
We use cloverage via kaocha to generate code coverage reports. Our CI service is setup to automatically generate then upload reports to CodeCov.
We have no specific goals for code coverage, but new code is generally expected to have tests.
So why measure coverage? It simply offers us some idea of what code our test suite hits.
We honor current and past contributors to rewrite-clj in our README file.
To update contributors, update doc/contributors.edn
then run:
clojure -M:update-readme
Can you improve this documentation? These fine people already did:
lread & Vincent CantinEdit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close