Building Middleware

This page is very incomplete and a work in progress.

Overview

This part of the documentation aims to guide you in the process of implementing nREPL middleware. We’ll cover the basics, the best practices and some of the common pitfalls you might encounter.

This page complements the middleware design documentation. Make sure you’re familiar with it before proceeding to implement any middleware.

Basics

The basic structure of most middleware is pretty simple:

An op dispatch function (a.k.a. handler) that intercepts requests to the middleware
Some functions that do the actual work for each op and provide the responses
A middleware descriptor that specifies which middlewares are provided/required and documents the API of the middleware

Optionally you can have some dynamic vars for configuration purposes. Below is a complete example:

(ns nrepl.middleware.completion
  "Code completion middleware."
  (:require
   [nrepl.util.completion :as complete]
   [nrepl.middleware :as middleware :refer [set-descriptor!]]
   [nrepl.misc :refer [response-for] :as misc]
   [nrepl.transport :as t])
  (:import nrepl.transport.Transport))

;; middleware configuration
(def ^:dynamic *complete-fn*
  "Function to use for completion. Takes three arguments: `prefix`, the completion prefix,
  `ns`, the namespace in which to look for completions, and `options`, a map of additional
  options for the completion function."
  complete/completions)

;; the business logic for the "completions" op
(defn completion-reply
  [{:keys [session prefix ns complete-fn options] :as msg}]
  (let [ns (if ns (symbol ns) (symbol (str (@session #'*ns*))))
        completion-fn (or (and complete-fn (misc/requiring-resolve (symbol complete-fn))) *complete-fn*)]
    (try
      (response-for msg {:status :done :completions (completion-fn prefix ns options)})
      (catch Exception e
        (response-for msg {:status #{:done :completion-error :namespace-not-found}})))))

;; the handler
(defn wrap-completion
  "Middleware that provides code completion.
  It understands the following params:

  * `prefix` - the prefix which to complete.
  * `ns`- the namespace in which to do completion. Defaults to `*ns*`.
  * `complete-fn` – a fully-qualified symbol naming a var whose function to use for
  completion. Must point to a function with signature [prefix ns options].
  * `options` – a map of options to pass to the completion function."
  [h]
  (fn [{:keys [op ^Transport transport] :as msg}]
    (if (= op "completions")
      (t/send transport (completion-reply msg))
      (h msg))))

(set-descriptor! #'wrap-completion
                 {:requires #{"clone"}
                  :expects #{}
                  :handles {"completions"
                            {:doc "Provides a list of completion candidates."
                             :requires {"prefix" "The prefix to complete."}
                             :optional {"ns" "The namespace in which we want to obtain completion candidates. Defaults to `*ns*`."
                                        "complete-fn" "The fully qualified name of a completion function to use instead of the default one (e.g. `my.ns/completion`)."
                                        "options" "A map of options supported by the completion function."}
                             :returns {"completions" "A list of completion candidates. Each candidate is a map with `:candidate` and `:type` keys. Vars also have a `:ns` key."}}}})

Not all middleware respond directly to ops. You can have middleware that simply modify requests or responses.

It’s important to make sure you don’t let some exceptions slide, as some clients might block when waiting for a response (as some clients will try to handle nREPL messages in a synchronous fashion due to various limitations).

Notice that you can have multiple statuses for one response - e.g. a combination of :done and some error status.

Also notice that the example middleware depends on the session middleware, as it’s fetching some data from the session. In practice this means that the session middleware has already been applied to the message for the completion middleware before the message made its way to it, and the required session data has been added to it.

Testing

Typically we’d be testing nREPL middleware with integration tests that generate middleware requests and check their responses. nREPL bundles a few helper functions to streamline the process. Here’s a simple example:

(ns nrepl.middleware.completion-test
  {:author "Bozhidar Batsov"}
  (:require
   [clojure.test :refer :all]
   [nrepl.core :as nrepl]
   [nrepl.core-test :refer [def-repl-test repl-server-fixture project-base-dir clean-response]])
  (:import
   (java.io File)))

(use-fixtures :each repl-server-fixture)

(defn dummy-completion [prefix _ns _options]
  [{:candidate prefix}])

(def-repl-test completions-op
  (let [result (-> (nrepl/message session {:op "completions" :prefix "map" :ns "clojure.core"})
                   nrepl/combine-responses
                   clean-response
                   (select-keys [:completions :status]))]
    (is (= #{:done} (:status result)))
    (is (not-empty (:completions result)))))

(def-repl-test completions-op-error
  (let [result (-> (nrepl/message session {:op "completions"})
                   nrepl/combine-responses
                   clean-response
                   (select-keys [:completions :status]))]
    (is (= #{:done :completion-error :namespace-not-found} (:status result)))))

(def-repl-test completions-op-custom-fn
  (let [result (-> (nrepl/message session {:op "completions" :prefix "map" :ns "clojure.core" :complete-fn "nrepl.middleware.completion-test/dummy-completion"})
                   nrepl/combine-responses
                   clean-response
                   (select-keys [:completions :status]))]
    (is (= #{:done} (:status result)))
    (is (= [{:candidate "map"}] (:completions result)))))

Best Practices

In this section we’ll go over some of the best practices for implementing nREPL middleware.

Code Structure

It’s best to keep the business logic decoupled from the middleware code and have middleware serve as a thin wrapper around it. This means that ideally you should have the business logic in a different namespace (or even a different library). A good example would be the completion middleware bundled with nREPL:

The actual completion logic lives in nrepl.util.completion
The middleware code lives in nrepl.middleware.completion and it simply delegates to nrepl.util.completion

This separation makes it easier to test the business logic in isolation and to reuse it outside of nREPL (a good example here would be the orchard library that cider-nrepl uses heavily).

As a corollary - you should avoid creating middleware that provides a lot of unrelated functionality. Ideally all ops within some middleware should be closely linked by their purpose.

Naming conventions

It’s recommended to prefix middleware names with wrap - e.g. wrap-complete, wrap-lookup, etc. This naming convention came from Ring middleware, which was a major influence on the design of nREPL.

nREPL itself breaks this convention with names like add-stdin and interruptible-eval, but those names are historical and hard to change at this point.

When it comes to ops, ideally their names should be verbs - e.g. complete, lookup, test, etc. Related ops can be grouped under some common prefix - e.g. test-var, test-ns, test-all.

It’s also prudent to prefix op names with some "namespace"-like prefix to avoid conflicts between different middleware - e.g. project-name/op-name.

That’s the reason why nREPL’s completion op is named completion instead of complete. cider-nrepl already has an op named complete and adding such an op to nREPL would have introduced some non-deterministic behaviour when it comes to the ordering of the two competing completion middleware. As a result, in some cases you’d be invoking nREPL’s op and in other cases cider-nrepl’s op. If `cider-nrepl had named the op cider/complete instead, that would have prevented this unfortunate situation.

Robustness and error handling

Given how misbehavior of a foundational tool like nREPL can greatly diminish the user experience, it is doubly important to prevent exceptions or errors to happen during middleware processing, but also vital to signalize that an error did occur (as the communication is often asynchronous, this needs to be done explicitly). A common pattern is to use nrepl.transport/safe-handle macro which accepts a list of op and handler pairs and wraps the handler invocation in try/catch block so that you don’t forget to. If an exception is thrown, the macro will send an error message through the message’s transport. For example:

(defn wrap-foo [h]
  (fn [msg]
    (safe-handle msg
      "foo" foo-reply
      :else h)))

This middleware will call (foo-reply msg) in a try/catch if the message has :op "foo", and otherwise will invoke the rest of the middleware chain with h. The return of (foo-reply msg) is merged with msg-specific fields and sent to the client. If (foo-reply msg) throws an exception, a message with :status #{:foo-error} will be sent to the client.

`Ctrl`+`k`	Jump to recent docs
`←`	Move to previous article
`→`	Move to next article
`Ctrl`+`/`	Jump to the search field