lsp4clj

A Language Server Protocol base for developing any LSP implementation in Clojure.

lsp4clj reads and writes from stdio, parsing JSON-RPC according to the LSP spec. It provides tools to allow server implementors to receive, process, and respond to any of the methods defined in the LSP spec, and to send their own requests and notifications to clients.

Usage

Create a server

To initialize a server that will read from stdin and write to stdout:

(lsp4clj.io-server/stdio-server)

The returned server will have a core.async :log-ch, from which you can read server logs (vectors beginning with a log level).

(async/go-loop []
  (when-let [[level & args] (async/<! (:log-ch server))]
    (apply logger/log level args)
    (recur)))

Receive messages

To receive messages from a client, lsp4clj defines a pair of multimethods, lsp4clj.server/receive-notification and lsp4clj.server/receive-request that dispatch on the method name (as defined by the LSP spec) of an incoming JSON-RPC message.

Server implementors should create defmethods for the messages they want to process. (Other methods will be logged and responded to with a generic "Method not found" response.)

These defmethods receive 3 arguments, the method name, a "context", and the params of the JSON-RPC request or notification object. The keys of the params will have been converted (recursively) to kebab-case keywords. Read on for an explanation of what a "context" is and how to set it.

;; a notification; return value is ignored
(defmethod lsp4clj.server/receive-notification "textDocument/didOpen"
  [_ context {:keys [text-document]}]
  (handler/did-open context (:uri text-document) (:text text-document))
  
;; a request; return value is converted to a response
(defmethod lsp4clj.server/receive-request "textDocument/definition"
  [_ context params]
  (->> params
       (handler/definition context)
       (conform-or-log ::coercer/location)))

The return value of requests will be converted to camelCase json and returned to the client. If the return value looks like {:error ...}, it is assumed to indicate an error response, and the ... part will be set as the error of a JSON-RPC error object. It is up to you to conform the ... object (by giving it a code, message, and data.) Otherwise, the entire return value will be set as the result of a JSON-RPC response object. (Message ids are handled internally by lsp4clj.)

Async requests

lsp4clj passes the language server the client's messages one at a time. It won't provide another message until it receives a result from the multimethods. Therefore, by default, requests and notifications are processed in series.

However, it's possible to calculate requests in parallel (though not notifications). If the language server wants a request to be calculated in parallel with others, it should return a java.util.concurrent.CompletableFuture, possibly created with promesa.core/future, from lsp4clj.server/receive-request. lsp4clj will arrange for the result of this future to be returned to the client when it resolves. In the meantime, lsp4clj will continue passing the client's messages to the language server. The language server can control the number of simultaneous messages by setting the parallelism of the CompletableFutures' executor.

Cancelled inbound requests

Clients sometimes send $/cancelRequest notifications to indicate they're no longer interested in a request. If the request is being calculated in series, lsp4clj won't see the cancellation notification until after the response is already generated, so it's not possible to cancel requests that are processed in series.

But clients can cancel requests that are processed in parallel. In these cases lsp4clj will cancel the future and return a message to the client acknowledging the cancellation. Because of the design of CompletableFuture, cancellation can mean one of two things. If the executor hasn't started the thread that is calculating the value of the future (perhaps because the executor's thread pool is full), it won't be started. But if there is already a thread calculating the value, the thread won't be interupted. See the documentation for CompletableFuture for an explanation of why this is so.

Nevertheless, lsp4clj gives language servers a tool to abort cancelled requests. In the request's context, there will be a key :lsp4clj.server/req-cancelled? that can be dereffed to check if the request has been cancelled. If it has, then the language server can abort whatever it is doing. If it fails to abort, there are no consequences except that it will do more work than necessary.

(defmethod lsp4clj.server/receive-request "textDocument/semanticTokens/full"
  [_ {:keys [:lsp4clj.server/req-cancelled?] :as context} params]
  (promesa.core/future
    ;; client may cancel request while we are waiting for analysis
    (wait-for-analysis context)
    (when-not @req-cancelled?
      (handler/semantic-tokens-full context params))))

Send messages

Servers also send their own requests and notifications to a client. To send a notification, call lsp4clj.server/send-notification.

(->> {:message message
      :type type
      :extra extra}
     (conform-or-log ::coercer/show-message)
     (lsp4clj.server/send-notification server "window/showMessage"))

Sending a request is similar, with lsp4clj.server/send-request. This method returns a request object which may be dereffed to get the client's response. Most of the time you will want to call lsp4clj.server/deref-or-cancel, which will send a $/cancelRequest to the client if a timeout is reached before the client responds.

(let [request (->> {:edit edit}
                   (conform-or-log ::coercer/workspace-edit-params)
                   (lsp4clj.server/send-request server "workspace/applyEdit"))
      response (lsp4clj.server/deref-or-cancel request 10e3 ::timeout)]
  (if (= ::timeout response)
    (logger/error "No reponse from client after 10 seconds.")
    response))

The request object presents the same interface as future. It responds to future-cancel (which also sends $/cancelRequest), realized?, future?, future-done? and future-cancelled?.

If the request is cancelled, later invocations of deref will return :lsp4clj.server/cancelled.

$/cancelRequest is sent only once, although lsp4clj.server/deref-or-cancel or future-cancel can be called multiple times.

Start and stop a server

The last step is to start the server you created earlier. Use lsp4clj.server/start. This method accepts two arguments, the server and a "context".

The context should be associative?. Whatever you provide in the context will be passed as the second argument to the notification and request defmethods you defined earlier. This is a convenient way to make components of your system available to those methods without definining global constants. Often the context will include the server itself so that you can initiate outbound requests and notifications in reaction to inbound messages. lsp4clj reserves the right to add its own data to the context, using keys namespaced with :lsp4clj.server/....

(lsp4clj.server/start server {:custom-settings custom-settings, :logger logger})

The return of start is a promise that will resolve to :done when the server shuts down, which can happen in a few ways.

First, if the server's input is closed, it will shut down too. Second, if you call lsp4clj.server/shutdown on it, it will shut down.

When a server shuts down it stops reading input, finishes processing the messages it has in flight, and then closes is output. Finally it closes its :log-ch and :trace-ch. As such, it should probably not be shut down until the LSP exit notification (as opposed to the shutdown request) to ensure all messages are received. lsp4clj.server/shutdown will not return until all messages have been processed, or until 10 seconds have passed, whichever happens sooner. It will return :done in the first case and :timeout in the second.

Socket server

The stdio-server is the most commonly used, but the library also provides a lsp4clj.socket-server/server.

(lsp4clj.socket-server/server {:port 61235})

This will start listening on the provided port, blocking until a client makes a connection. When the connection is made it returns a lsp4clj server that has the same behavior as a stdio-server, except that messages are exchanged over the socket. When the server is shut down, the connection will be closed.

Development details

Tracing

As you are implementing, you may want to trace incoming and outgoing messages. Initialize the server with :trace-level "verbose" and then read traces (two element vectors, beginning with the log level :debug and ending with a string, the trace itself) off its :trace-ch.

(let [server (lsp4clj.io-server/stdio-server {:trace-level "verbose"})]
  (async/go-loop []
    (when-let [[level trace] (async/<! (:trace-ch server))]
      (logger/log level trace)
      (recur)))
  (lsp4clj.server/start server context))

:trace-level can be set to "off" (no tracing), "messages" (to show just the message time, method, id and direction), or "verbose" (to also show details of the message body).

The trace level can be changed during the life of a server by calling, for example, (ls4clj.server/set-trace-level server "messages"). This can be used to respect a trace level received at runtime, either in an initialize request or a $/setTrace notification.

Testing

A client is in many ways like a server—it also sends and receives requests and notifications and receives responses. That is, LSP uses JSON-RPC as a bi-directional protocol. As such, you may be able to use some of lsp4clj's tools to build a mock client for testing. See integration.client in clojure-lsp for one such example.

You may also find lsp4clj.server/chan-server a useful alternative to stdio-server. This server reads and writes off channels, instead of stdio streams. See lsp4clj.server-test for many examples of interacting with such a server.

Caveats

You must not print to stdout while a stdio-server is running. This will corrupt its output stream and clients will receive malformed messages. To protect a block of code from writing to stdout, wrap it with lsp4clj.server/discarding-stdout. The receive-notification and receive-request multimethods are already protected this way, but tasks started outside of these multimethods need this protection added. Consider using a lsp4clj.socket-server/server to avoid this problem.

Known lsp4clj users

• clojure-lsp: A Clojure LSP server implementation.