Readme — com.github.gnl/playback 0.5.0

Interactive Programming and Print Debugging Reimagined

Playback provides immediate, frictionless visibility into a function’s dataflow; makes it easy to call and replay with real application input without the need for mocking; enables the hassle-free extraction of any data observed in the process; and makes possible an exceptionally tight dev loop with instant feedback.

It does so out of the box with a minimal number of intuitive, unintrusive, zero-config operations. Two and a half reader tags, to be exact:

#>   ; trace output + replay function
#>>  ; trace output and input/bindings/steps (depending on the form) + replay function
#><  ; reference traced data

Quick Start and Walkthrough

Add to your dev (and only dev) dependencies.

Load the playback.preload namespace on application launch and the Portal window should pop right up.

Clojure:

dev/user.clj

(ns user
  (:require [playback.preload]))

ClojureScript:
shadow-cljs.edn
```
{:builds {:app {:devtools {:preloads [playback.preload]}}}}
```
See the Shadow or ClojureScript docs for more details.

If you want to tweak the Portal configuration you can use your own preload namespace by copying playback.preload to your project’s source path, renaming it accordingly, and passing the config map to playback.core/open-portal!.

#> traces the evaluated output of any code and sends the data to Portal;
#>> traces input or intermediate data as well, depending on the form (let bindings, threading macro steps, defn/fn arguments, loop/recur bindings, etc.)
Traced functions – #>(defn, #>(defmethod, #>(>defn – cache their most recent input and are automatically called with it (= replayed) whenever they’re reloaded by your environment’s send-to-REPL or recompile-on-save functionality.

This way you can have a function receive some real application data, make changes to the code, eval/reload it and instantly see the updated dataflow, output and possible spec failures as it’s auto-replayed and retraced with the same input.

Playback does not auto-replay functions whose names end with !, so make sure to name your STM-unsafe functions as the Gods intended.

Use additional nested #> or #>> tags inside the function to zoom in on the specific code you’re interested in, similar to the way you would use print statements.

#>< _ references the currently selected data in the Portal window. You can take any previously traced data, feed it to another function, play around with it in the REPL and #> the results back to Portal to inspect them.

The _ in #>< _ is just a random placeholder, because Clojure reader tags have to be applied to a form. You can use anything in its place and it will be replaced with the selected Portal data. I like #><[].

You can trace and replay re-frame handlers and subscriptions just like regular functions, without having to extract the anonymous fn into a top-level defn:

#>>(reg-event-fx
    ::boot
    (fn [_ _]
      {:db default-db}))

If you want to trace/replay your own compatible macros with Playback, you can extend its existing optypes like this:

dev/user.clj

(ns user
  (:require [playback.core :as playback]))

(playback/extend-optypes {::playback/defn   ['foo.bar/defn 'foo.bar/defn-]
                          ::playback/fn-reg ['foo.bar/reg-sub 'foo.bar/reg-fx]})

See optype→ops in playback.core for a list.

If you want to display some data with a particular Portal viewer, you can define some helper functions in your `user.clj(s) like this:

dev/user.clj(s)

(ns user)

(defn tree
  [expr]
  (with-meta expr {:portal.viewer/default :portal.viewer/tree}))

(defn table
  [expr]
  (with-meta expr {:portal.viewer/default :portal.viewer/table}))

You can then wrap any traced expression with #>> (user/table …). See the Portal documentation for more details.

Playback works great with Fulcrologic’s indispensable Guardrails (or its now discontinued predecessor Ghostwheel). If you set {:tap>? true} in its configuration, you can see the results of failed spec checks in Portal right inside the traced dataflow (though you’ll probably still want to take a look at the REPL or console for the human-readable error message). A similar workflow should be possible with Malli function schemas and instrumentation as well.

Instant re-render on eval

To get faster feedback and more control over hot-reloading, it’s recommended that you disable autobuild and instead use your editor’s send-top-form-to-REPL functionality in combination with Playback’s refresh-on-eval middleware to manually reload individual functions.

Add the middleware:
shadow-cljs.edn
```
{:nrepl {:middleware [playback.nrepl-middleware/refresh-on-eval]
         :port       9000}}
```
See the Shadow documentation for more details or the nREPL one for info on other build tools.

Initialise it on launch:

dev/user.clj (← not .cljs)

(ns user
  (:require [playback.nrepl-middleware :as middleware]))

(middleware/init-refresh-on-eval!
 ;; Refresh/re-render functions to call post-reload
 ['gnl.clojure-playground.main/mount-root]
 ;; Namespace prefixes in which eval triggers a refresh
 ["gnl.clojure-playground"])

Disable autobuild:

Shadow REPL

;; In the Clojure REPL, before starting the ClojureScript one with `(shadow/repl :app)`:
(shadow/watch-set-autobuild! :app false)
;; To trigger a manual recompile:
(shadow/watch-compile! :app)

By default, refresh-on-eval is disabled for traced functions, the idea being that you would usually mess around in the code, repeatedly sending it to the REPL to replay and watch the dataflow in the trace, rinse and repeat until it works, and only then would you remove the #> tag, reload and have the application re-render. You can change this behaviour with (middleware/set-refresh-on-traced-fn! true).

If you are using a Clojure REPL in a namespace with a refresh-enabled prefix meant for ClojureScript, the middleware will try to call the likely non-existent Clojure equivalent of the re-render function and throw an exception. The simplest solution is to create a noop function with the same name that doesn’t do anything.

On using (unqualified) reader tags

Unqualified, non-namespaced reader tags are reserved for Clojure and their usage by anyone else is frowned upon by the powers that be, and for a good reason. That being said, I went ahead, did it anyway and – in the time-honoured tradition of everyone who ever thought they knew better while not being in charge – chose to ask for forgiveness rather than permission. This is why:

Given that Playback is meant to be used continuously as a fundamental part of a Clojurian’s dev workflow and is trying to challenge the ubiquity of print debugging, it has to be dead simple. Every extra character that needs typing or reading adds friction.
When using macros instead of reader tags one has to add :require and :refer directives to debug and then remove them again before pushing commits or alternatively leave them in and use noop/stub namespaces and artifacts in the production build (or just leave it all in there and cross one’s fingers that no forgotten performance-killing or security-impacting debug statements slip into prod). Way too much complexity, friction and clutter for something that wants to replace and improve upon print.
#> tags aren’t meant to become a permanent part of the codebase – just like print debugging statements – so changing the syntax in the future, should it become necessary, comes at a very limited cost. In the worst-case scenario that Clojure does at some point introduce conflicting reader tags, I’ll be forced to grudgingly update Playback and its users will be forced to go through a brief period of mild discomfort as they retrain their muscle memory to the new tags. But while this outcome is not beyond the realm of possibility, it doesn’t appear particularly imminent or at all likely.
And last but definitely not least – with a bit of imagination #> kind of looks like a play button, while #>< somewhat resembles a portal, and giving up this kind of perceived semiotic perfection would greatly displease me.

The Road to 1.0

…in no particular order:

Add babashka support
Add/complete support for re-frame handlers, subscriptions and other common function-like constructs and function registrations to have it all work transparently just like tracing/replaying a regular function, without requiring the user to do any kind of refactoring to accommodate Playback.
Specs
Tests
Add support for all debux features (transducers, …)
Add support for electric
Think about how to handle the replay of side-effectful, STM-unsafe functions without setting things on fire
Node support
Consider switching to jpmonettas/hansel for the underlying instrumentation/tracing implementation

Contributions and Support

I’m always open to PRs, but please do reach out first if you want to tackle something bigger so we can make sure we’re on the same page.

Other than that, if you or your company have benefitted professionally from my open-source work or would simply like to support further development and can afford it, your GitHub sponsorship would be much appreciated:

Become a Champion of the Lisp Arts

General inquiries as to my availability for paid work, open source or otherwise, are welcome.

Acknowledgements, Prior Art and Rationale

First the obligatory disclaimer that Playback stands on the shoulders of giants – those being Philos Kim’s debux and Chris Badahdah’s portal in particular – and mostly just does some dot-connecting and magic-sprinkling on top in order to fuse them into what is hopefully a highly enjoyable interactive development experience, for which, as my small contribution to the never-ending abuse of the REPL acronym, I would like to propose the term RETL, as in Read–Eval–Trace Loop.

The idea to re-render on eval was stolen from Misha Karpenko’s nREPL experiments; Spellhouse’s Clairvoyant and Day8’s re-frame tracer were the initial inspiration for and the foundation of Ghostwheel’s tracing functionality which was a first shaky step towards what I imagined REPL-based development and debugging should more or less look like. The corresponding section of the original omnibus project’s README is a good summary of the evolving vision that Playback is a part of.

Juan Monetta’s FlowStorm is a fantastic tracing debugger that fits perfectly within this vision, but appears to occupy a somewhat different category than Playback – one in which a certain level of (relative) complexity is considered a reasonable trade-off for maximum capability. Playback meanwhile aims to extract the highest possible amount of power from the constraints of not exceeding the complexity of print. I believe it actually manages to be even simpler than that and is therefore not a trade-off. Depending on the situation, sometimes exchanging simplicity for power is worth it and sometimes it is not – and Playback’s success as a debugging tool is measured by whether you instinctively reach for it instead of print in the latter case.

But to look at it as just a type of debugger, tracer or dataflow inspector is to sell it short. In combination with Guardrails or Malli in particular, it provides instant, precise feedback on the type, content and rendering of real application data repeatedly flowing through a function as it changes iteratively in a tight, low-latency dev loop largely free of many of the common challenges and pitfalls of REPL workflows or dynamically typed languages in general, for that matter. It reduces the extensive amount of mental code compilation and execution that developers commonly perform in their heads, by a significant enough amount that it can be reasonably considered to be a different, and better, paradigm, one that gets much closer to fulfilling the interactive programming promise that classical REPL-based development often fails to deliver on.

I believe we have some low-hanging Clojure fruit to pick here and this is the way.

As always, go boldly forth, fellow maker, create freely and be not afraid of a messy road.