Merging rewrite-clj and rewrite-cljs

I’ve used diff-apis to compare apis. Normally I would have excluded any apis tagged with :no-doc metadata, but because many folks used undocumented features in rewrite-clj v0 and rewrite-cljs, I have done a complete comparison of all publics - except where noted. Each report contains some observations under the "Notes" header.

No ability to read from files when using rewrite-clj v1 from ClojureScript.

I have moved from leiningen to tools cli and deps.edn. Like everything, this change has pros and cons. Overall, I like the simplicity and control it brings. Babashka scripts take the place of lein aliases where I can have the build do exactly what I want it to.

The future of Travis CI looked a bit tenuous when I started work on rewrite-clj v1. I initially switched over to CircleCI, but then when GitHub Actions became available decided it was a better fit:

After looking around, I settled on the following for continuous integration:

During development, I found the following helpful:

I see plenty of version scheme variations out there these days. Here are a few examples I find interesting:

Rewrite-clj v1 is not a new project. I feel the version should reflect at least some familiarity with its v0 scheme.

As of this writing the current version of rewrite-clj is 0.6.1. I am guessing that the 0 is an unused version element, and we have a 0.major.minor scheme.

Rewrite-clj v1 is going to switch to a major.minor.<commit count>-<qualifier> scheme.
Our first version will be 1.0.451-alpha where 451 is just a wild guess right now.

An small awkwardness with this scheme is the change log. The change log should be part of the release but it does reference a git commit count. This will be addressed by automatically updating the change log doc with the release version as part of the release process.

We’ll opt not to make SNAPSHOT releases and assume the community is good with testing pre-releases via GitHub coordinates. We can adapt if there is a real need for SNAPSHOT releases.

We’ll keep a CHANGELOG.adoc carried on from rewrite-clj v0’s CHANGES.md.

Release cadence will be as needed. I don’t want us to feel precious about releases. If there is a benefit to cutting a new release with a small change or fix, even just to docs, we’ll go ahead and do it.

When I first started to experiment with a cljc version of rewrite-clj, my directory layout looked like:

After a certain amount of work, I realized the majority of the code was cljc so opted for the much simpler:

Some command line tools written in Clojure are using Graal to compile to native executables for fast startup times.

Others have done the work to test that rewrite-clj v0 can be compiled with Graal. There is benefit to the community to test that rewrite-clj v1 can also be compiled to native code with Graal.

Noticing that there were differing approaches Graalifying Clojure, none of them centrally documented, @borkdude and I created clj-graal-docs to develop and share scripts and tips.

My goal is to run the rewrite-clj v1 test suite from a GraalVM native image to give some confidence that rewrite-clj v1 works after compiled with Graal.

Borkdude is kind enough to ping me when there are issues with the internally forked version of rewrite-clj he uses for clj-kondo. It turns out that clojure.tools.reader.edn does not parse garden-style keywords such as :&::before. The reader sees a double colon as illegal if it is anywhere in the keyword. Borkdude overcame this limitation by allowing a keyword to contain embedded double colons via a customized version of clojure.tools.reader.edn's read-keyword function.

I transcribed his work to rewrite-clj v1.

The maintenance cost to hacking a 3rd party lib is that upgrades will have to be carefully tracked. That said, we do have a good suite of tests that should uncover any issues.

While Clojure reads 'org/clojure/math.numeric-tower, clojure.tools.reader.edn barfs on this and therefore rewrite-clj does as well.

It has been documented as illegal for a symbol to have more than one /.

I have opted to not, at this time, adapt rewrite-clj v1 to allow parsing of this illegal syntax. This might seem a bit hypocritical because I did, some time ago, innocently raise an issue on clj-kondo for this.

ClojureScript uses Google Closure under the hood. Because of the way Google Closure handles namespaces, some namespaces that work fine on Clojure clash under ClojureScript. Some rewrite-clj v0 namespaces clash for ClojureScript, for example:

The original rewrite-cljs author worked around this problem by renaming namespaces to avoid the clashes.

None of these namespaces are part of public APIs, but because I see a lot of code that uses these internal namespaces, I decided to preserve the existing rewrite-clj v0 and rewrite-cljs naming for rewrite-clj v1.

rewrite-clj v0 purposefully only generated documentation for specific namespaces. It is reasonable to assume that these namespaces represent the public API:

I am not sure why rewrite-clj.custom-zipper is included in the documented public API, because its functionality is exposed through rewrite-clj.zip, I expect this was perhaps an oversight, but might be wrong.

Because what is public versus what is private was not stressed strongly in the rewrite-clj v0 README, I frequently see private APIs used in code. For this reason, I’ve worked, within reason, not to break what I understand to be private APIs.

rewrite-clj allows parsed Clojure/ClojureScript/EDN to be converted back and forth to s-expressions. Example from a REPL session:

While I expect this can be quite convenient, it does come with caveats:

My guidance is use sexpr in only in specific cases, where you know ahead of time what you are parsing. General blind use of sexpr is not recommended.

For rewrite-clj v1 itself, I have removed internal problematic uses of sepxr and documented some of its nuances.

Rewrite-clj makes use of Clojure’s reader. There are a few choices though:

As I understand it, clojure.tools.reader.edn is the safest choice and rewrite-clj v1 uses it in all cases.

Rewrite-clj v0 makes use of a slightly modified version of Potemkin import-vars. The intent of import-vars is to make it easy to expose a public API from a set of internal namespaces.

When I first reviewed its usage in rewrite-clj, I found import-vars to be quite elegant. I have since learned that there is quite a bit of strong opinion in the Clojure community surrounding import-vars. Not all of it is rosy.

Also, there is no ClojureScript version of import-vars.

Rewrite-clj v0 used a customized version for potemkin defprotocol+. It could be that I missed something, but I did not see how it would benefit rewrite-clj v1. In the spirit of simplifying a cljc code-base, I turfed defprotocol+ in favour of plain old defprotocol.

We can reintroduce defprotocol+ if we learn that it does actually help with performance significantly.

Rewrite-clj v0:

Rewrite-cljs:

Because the positional support in rewrite-clj v0 tracks row/col even after zipper modifications, we use it in rewrite-clj v1 instead of rewrite-cljs’s implementation. We:

The most glaring breaking change for ClojureScript is that it must now create the zipper with positional support enabled, for example: (z/of-string "[1 2 3]" {:track-position true})