Introduction

TL;DR: revolt is a plugins/tasks oriented library which makes it easier to integrate beloved dev tools like nrepl, rebel readline or clojurescript into application, based on Cognitect's command line tools.

To see it in action look at revolt-edge example.

Clojure projects historically leverage the power of two glorious build tools: leiningen and boot. Both battle-tested, feature rich alternatives allow to choose either declarative (leiningen) or fully programmable way to manage with tons of dependencies, dev / prod builds and whole bunch of tasks crucial for clojure developer.

The choice was relatively easy. Up util recent.

One day, according to their tradition, Cognitect surprisingly announced a new player on the stage - a command line tools for running Clojure programs (Deps and CLI Guide). It was not only about a new tool, the most significant improvement presented to community was entirely new way of working with dependencies. Things like multi-module project with separate dependencies and dependencies stright from git repo became possible just right out of the box.

Despite of this awesomeness people started wondering how to join all these toys together. Drop well known lein/boot and go along with Cognitect's way or to use new deps for application-specific dependencies and still leverage boot/lein tools for development only? Or maybe ignore new kid on the block and stick with bullet-proof tools we used for ages?

One of most interesting moves within this area was JUXT's edge and their attempt to build a simple but complete Clojure project based on the newest and most experimental things found around. Yeah, including Cognitect's new dependencies.

Revolt is inspired by JUXT's edge and, at its core, tries to simplify attaching all these shiny tools to the project by gathering them in form of tasks and plugins. And yes, it depends on Cognitect's dependencies underneath and makes heavy use of newly introduced aliases by the way.

Doesn't it sound like a Lisp Curse again? :)

What's in the box?

A couple of plugins you may really like:

• [x] Rebel REPL to give you best REPL experience
• [x] Figwheel must-have for web development
• [x] nREPL obviously to let you use Emacs and Cider
• [x] Filesystem watcher able to watch and react on files changes

and a few built-in tasks:

• [x] scss - transforms scss files into css
• [x] cljs - a cljs compiler
• [x] aot - ahead-of-time compilation
• [x] jar - jar packager
• [x] test - clojure.test runner based on Metosin's bat-test
• [x] info - project info (name, description, package, version, git branch, sha...)
• [x] assets - static assets fingerprinting
• [x] capsule - capsule packaging
• [x] codox - API documentation with codox

External tasks planned:

• [x] migrations - flyway based database migrations
• [x] catapulte - jar files installer/deployer
• [ ] lint - linter based on eastwood
• [ ] analyse - static code analyzer based on kibit
• [ ] ancient - looking for outdated dependencies

Plugins

Plugins are these guys who always cause problems. No matter if that's boot or lein, they just barely fit into architecture with what they do. And they do a lot of weird things, eg. nREPL is a socket server waiting for connection, REPL is a command line waiting for input, watcher on the other hand is a never ending loop watching for file changes. Apples and oranges put together into same basket.

Revolt does not try to unify them and pretend they're same tasks as cljs compilation or scss transformation. They are simply a piece of code which starts when asked and stops working when JVM shuts down. Nothing more than that. Technically, plugins (as well as tasks) are identified by qualified keyword and configured in a separate file. Typical configuration looks like following:

{:revolt.plugin/nrepl    {:port 5600}
 :revolt.plugin/rebel    {:init-ns "foo.system"}
 :revolt.plugin/figwheel {:builds ["main"]}
 :revolt.plugin/watch    {:excluded-paths ["src/clj"]
                          :on-change {:revolt.task/sass "glob:assets/styles/*.scss"}}}

Right after activation plugins usually stay in a separate thread until deactivation phase hits them in a back which happens on JVM shutdown, triggered for example when plugin running in a main thread (like rebel) gets interrupted.

Plugins love to delegate their job down to someone else, as all those bad guys do. In our cruel world these are tasks who handle most of ungrateful work on behalf of Master Plugins. As an example: watch plugin observes changes in a filesytem and calls a sass task when *.scss file is altered. Sometimes, task has to be explicitly configured to have plugin working, as it takes place for example in figwheel case which needs cljs task configured to run.

Ok, but how to specify which plugins do we want to activate? This is where clj tool from Cognitect comes onto scene, but more on that a bit later...

Tasks

If we called plugins as "bad guys", tasks are definitely the opposite - kind of little dwarfs who are specialized to do one job and do it well. And similar to plugins, there is a bunch of built-in tasks ready to serve you and take care of building and packaging your application. Oh, and they can generate documentation too.

To understand how tasks work, imagine them as a chain of dwarfs, each of them doing specific job and passing result to the next one:

clean ⇒ info ⇒ sass ⇒ cljs ⇒ capsule

which can expressed as a composition:

(capsule (cljs (sass (info (clean)))))

or in a bit more clojurey way:

(def build (comp capsule cljs sass info clean))

This way calling a build composition will clean a target directory, generate project information (name, package, version, git sha...), generate CSSes and finally pack everything into an uberjar (a capsule actually). Each of these tasks may generate intermediate result and pass it as a map to the next one in a context, eg. info task gathers project related information which is at the end passed to capsule which in turn makes use of these bits to generate a correct package.

To have even more fun, each task can be pre-configured in a very similar way as plugins are:

:revolt.task/info  {:name "foo"
                    :package bar.bazz
                    :version "0.0.1"
                    :description "My awesome project"}

:revolt.task/test  {:report :pretty}

:revolt.task/sass  {:source-path "assets/styles"
                    :output-path "styles"}

:revolt.task/codox {:source-paths ["src/clj"]
                    :source-uri "http://github.com/fuser/foo/blob/{version}/{filepath}#L{line}"
                    :namespaces [foo.main foo.core]}

:revolt.task/cljs  {:compiler {:optimizations :none
                               :output-dir "scripts/out"
                               :asset-path "/scripts/core"
                               :preloads [devtools.preload]}
                    :builds [{:id "main-ui"
                              :source-paths ["src/cljs"]
                              :compiler {:main "foo.main"
                                         :output-to "scripts/main.js"}}]}

:revolt.task/assets {:assets-paths ["assets"]}

:revolt.task/capsule {:exclude-paths #{"test" "src/cljs"}
                      :output-jar "dist/foo.jar"
                      :capsule-type :fat
                      :main "foo.main"
                      :min-java-version "1.8.0"
                      :jvm-args "-server"
                      :caplets {"MavenCapsule" [["Repositories" "central clojars(https://repo.clojars.org/)"]
                                                ["Allow-Snapshots" "true"]]}}

Let's talk about task arguments now.

Having tasks configured doesn't mean they are sealed and can't be extended in current REPL session any more. Let's look at the sass task as an example. Although it generates CSSes based on configured :source-path, as all other tasks this one also accepts an argument which can be one of following types:

• A keyword. This type of arguments is automatically handled by revolt. As for now only :help responds - returns a human readable description of given task.
• A java.nio.file.Path. This type of arguments is also automatically handled by revolt and is considered as a notification that particular file has been changed and task should react upon. sass task uses path to filter already configured :resources and rebuilds only a subset of SCSSes (if possible).
• A map. Here it's up to task how to handle this kind of argument, by convension revolt simply merges provided map into existing configuration:

(info {:environment :testing})
⇒ {:name "foo", :package "bar.bazz", :version "0.0.1", :description "My awesome project", :environment :testing}

Sometimes, in particular when tasks are composed together, it may be useful to provide argument with help of partial function:

(def build (comp capsule cljs sass (partial info {:environment :testing}) clean))

This way we can tinker with our builds and packaging in a REPL without changing a single line of base configuration. Eg. to generate a thin capsule (where dependencies will be fetched on first run) with an heavy-optimized version of our clojurescripts, we can construct a following pipeline:

(def build (comp (partial capsule {:capsule-type :thin})
                 (partial cljs {:compiler {:optimizations :advanced}})
                 sass
                 info
                 clean))

Now, as we know already how tasks work in general and how additional argument may extend or alter their base configuration, the question is how can we get these precious tasks into our hands?

Well, quite easy actually. As mentioned before, tasks are denoted by qualified keywords like :revolt.task/capsule. All we need is now to require-a-task :

(require '[revolt.task :as t])  ;; we need a task namespace first
(t/require-task ::t/capsule)    ;; now we can require specific task

(capsule)                       ;; task has been interned into current namespace
⇒ {:uberjar "dist/foo.jar"}

Indeed, require-task is a macro which does the magic, it loads and interns into current namespace required task. It's also possible to intern a task with different name:

(t/require-task ::t/capsule :as caps)  ;; note the ":as caps" here

(caps)
⇒ {:uberjar "dist/foo.jar"}

or to save unnecessary typing and load a bunch of tasks at once with require-all macro:

(t/require-all [::t/clean ::t/cljs ::t/sass ::t/capsule ::t/aot ::t/info])
(def build (comp capsule cljs sass info clean))

(build)
⇒ { final context is returned here }

require-task and require-all are simple ways to dynamically load tasks we want to play with and by chance turn our REPLs into training ground where all tasks are impatiently waiting to be used and abused :)

Following is the complete list of built-in tasks:

task	description	parameters
clean	cleans target directory	:extra-paths collection of additional paths to clean
sass	converts sass/scss assets into CSS	:source-path directory with sass/scss resources to transform :output-path directory where to store generated CSSes :sass-options additional sass-compiler options: ⇒ :source-map (bool) Enable source-maps for compiled CSS ⇒ :output-style :nested, :compact, :expanded or :compressed
assets	fingerprints static assets like images, scripts or styles	:assets-path collection of paths with assets to fingerprint :exclude-paths collection of paths to exclude from fingerprinting :update-with-exts extensions of files to update with new references to fingerprinted assets By default all javascripts, stylesheets and HTML resources are scanned for references to fingerprinted assets. Any recognized reference is being replaced with fingerprinted version.
aot	Ahead-Of-Time compilation	:extra-namespaces collection of additional namespaces to compile
jar	JAR file packager	:exclude-paths collection of paths to exclude from jar package :output-jar path to output jar (dist/-.jar by default)
cljs	clojurescript compiler	:compiler global clojurescript compiler options used for all builds :builds collection of builds, where each build consists of: ⇒ :id build identifier ⇒ :source-paths project-relative path of clojurescript files to compile ⇒ :compiler - clojurescript compiler options
test	clojure.test runner	:notify enable sound notification? (defaults to true) :parallel run tests in parallel? (defaults to false) :test-matcher regex to select test ns-es (defaults to #".*test") :report reporting function (:pretty, :progress or :junit) :filter fn to filter the test vars :on-start fn to call before running tests (after reloading namespaces) :on-end fn to call after running tests :cloverage enable Cloverage coverage report? (defaults to false) :cloverage-opts Cloverage options (defaults to nil)
codox	API documentation generator	:name project name, eg. "edge" :package symbol describing project package, eg. defunkt.edge :version project version, eg. "1.2.0" :description project description to be shown :namespaces collection of ns-es to document (by default all ns-es)
info	generates project information	:name project name, eg. "edge" :package symbol describing project package, eg. defunkt.edge :version project version, eg. "1.2.0"
capsule	generates an uberjar-like capsule	:capsule-type capsule type: :empty, :thin or :fat (defaults to :fat) :exclude-paths collection of project paths to exclude from capsule :output-jar project related path of output jar, eg. dist/foo.jar :main - main class to be run Capsule options: :min-java-version :min-update-version :java-version :jdk-required? :jvm-args :environment-variables :system-properties :security-manager :security-policy :security-policy-appended :java-agents :native-agents :native-dependencies :capsule-log-level

Usage

Ok, so having now both plugins and tasks at our disposal, let's get back to the question how clj tool can make use of these toys. Clj comes with a nice mechanism of aliases which allows to specify at command line additional dependencies or classpaths to be resolved when application starts up. Let's add few aliases to group dependencies based on tools required during development time: Assuming clojurescript, nrepl and capsule for packaging as base tools being used, this is all we need in deps.edn:

{:aliases {:dev {:extra-deps  {defunkt/revolt {:mvn/version "1.3.0"}}
                 :extra-paths ["target/assets"]
                 :main-opts   ["-m" "revolt.bootstrap"
                               "-p" "nrepl,rebel"]}

           ;; dependencies for nrepl
           :dev/nrepl {:extra-deps {cider/cider-nrepl {:mvn/version "0.21.0"}
                                    refactor-nrepl {:mvn/version "2.4.0"}}}

           ;; dependencies for clojurescript
           :dev/cljs {:extra-deps {org.clojure/clojurescript {:mvn/version "1.10.238"}
                                   binaryage/devtools {:mvn/version "0.9.9"}
                                   com.bhauman/figwheel-main {:mvn/version "0.1.9-SNAPSHOT"}
                                   re-frame {:mvn/version "0.10.5"}
                                   reagent {:mvn/version "0.8.0-alpha2"}}}

           ;; dependencies for packaging tasks
           :dev/pack {:extra-deps {co.paralleluniverse/capsule {:mvn/version "1.0.3"}
                                   co.paralleluniverse/capsule-maven {:mvn/version "1.0.3"}}}}}

Note the :extra-paths and :main-opts. First one declares additional class path - a target/assets directory where certain tasks (eg. sass, cljs, aot) dump their generated assets like CSSes or compiled clojurescripts. This is required only to keep things auto-reloadable - application needs to be aware of resources being (re)generated.

:main-opts on the other hand are the parameters that clj will use to bootstrap revolt: -m revolt.bootstrap instructs clj to use revolt.bootstrap namespace as a main class and pass rest of parameters over there.

Here is the complete list of all accepted parameters:

-c, --config     : location of configuration resource.

-d, --target-dir : location of target directory (relative to project location). This is where all re/generated
                   assets are being stored. Defaults to "target".

-p, --plugins    : comma separated list of plugins to activate. Each plugin (a stringified keyword) may be
                   specified with optional parameters:

                      clojure -A:dev:dev/nrepl:dev/cljs:dev/pack -p revolt.task/nrepl,revolt.task/rebel:init-ns=revolt.task
                  
-t, --tasks      : comma separated list of tasks to run. Simmilar to --plugins, each task (a stringified keyword)
                   may be specified with optional parameters:

                      clojure -A:dev:dev/nrepl:dev/cljs:dev/pack -t revolt.plugin/clean,revolt.plugin/info:env=test:version=1.1.2

To make things even easier to type, namespace parts of qualified keywords may be omitted when a built-in task or plugin is being used. So, it's perfectly valid to call:

                      clojure -A:dev:dev/nrepl:dev/cljs:dev/pack -t clean,info:env=test:version=1.1.2

Last thing to add - when running revolt both with --plugins and --tasks the latter takes precedence, which means that plugins get activated once all required tasks finish their work.

Development and more tech details

This is to describe in details of how plugins and task are loaded and initialized and provide a simple guide to develop own extensions.

Plugins rediscovered

When revolt starts up, it collects all the keywords listed in --plugins and sequentially loads corresponding namespaces calling a create-plugin multi-method (with keywords themselves as a dispatch values) at the end. Every such a function returns an object which extends a Plugin protocol:

    (defprotocol Plugin
      (activate [this ctx] "Activates plugin within given context")
      (deactivate [this ret] "Deactivates plugin"))

This simple mechanism allows to create a new plugin just like that:

    (ns defunkt.foo
      (:require [revolt.plugin :refer [Plugin create-plugin]]))
      
    (defmulti create-plugin ::bar [_ config]
      (reify Plugin
        (activate [this ctx] ...)
        (deactivate [this ret] ...)))

And configure it later as follows:

{:defunkt.foo/bar {:bazz 1}}

Each plugin gets a session context during activation phase. Context contains all the crucial stuff that most of plugins base on:

    (defprotocol SessionContext
      (classpaths [this]   "Returns project classpaths.")
      (target-dir [this]   "Returns a project target directory.")
      (config-val [this k] "Returns a value from configuration map."))

Note that plugin activation should return a value required to its correct deactivation. This value will be passed later to deactivate function as ret.

Tasks rediscovered

tbd.