nl.jomco/with-resources

A simple Clojure protocol for resource life cycle management

Problem statement

When writing software that interacts with the outside world, we often need to work with system resources, modeled by objects that have a life cycle.

Examples of resources are file handles (open files / streams), network ports / listeners, sockets, database connections and connection pools, web servers like jetty. These resources have a life-cycle; they are initialized at some point and must be cleaned up after use, to make sure that data is flushed to disk, to free up a port for a new listener etc.

On the JVM and JavaScript engines, resources that have a life cycle need to be managed explicitly; you cannot rely on garbage collection (GC) to clean up a resource when it's not used anymore, since GC is not guaranteed to happen at any specific moment (or at all). But managing resource life cycles is not straightforward in the face of dependency injection, exceptions, code reloading, name space refresh, POSIX Signals, JVM shutdown and REPL-driven development.

Rules of thumb

Afew rules of thumb can make reasoning about resources easier:

• Always put resources in a place (a local life cycle scope or a global).
• Collections that contain resources should be handled like resources.
• Do not close a resource you've not opened yourself.

Solution

This library aims to make it easier to follow good practice. In order to do that it provides:

• an extensible protocol for implementing resources that need life cycle management
• functions and macros defining life cycle scopes
• functions and macros defining resource collections

The Resource Protocol

In order to close resources, this library provides a Clojure Protocol (and associated Java interface) nl.jomco.resources.Resource, with a single method (close [resource]).

The close method should "stop" the resource and block until the resource is cleaned up. The close method may throw an exception.

This is very similar to java's java.util.AutoCloseable and java.io.Closeable interfaces, but because Resource is a Clojure protocol and not a plain Java interface, it can be extended to existing classes, interfaces and collections. Resource is extended to AutoCloseable, so any AutoCloseable or Closeable satisfies Resource.

Resource is also extended to java.lang.Object as a no-op, so any object may be used as a resource.

Extending Resource to existing Classes or Interfaces

(extend-protocol Resource
  org.eclipse.jetty.server.Server
  (close [server]
    (.stop server)))      ;; call Jetty's .stop method on close

Extending Resource to Clojure objects: closeable

Resource may be extended via metadata, meaning that Clojure IObj instances (so individual collections, records etc) can provide their own close method. The closeable function makes it easy to add close behavior to an object:

(require '[nl.jomco.resources :refer [close closeable]])

(defn my-cleanup-function
   [resource]
   ;; ... do some cleanup
   )

(with-resources [resource (closeable {:some "object"} my-cleanup-function)]
   ...
   (close resource))

See the Clojure Protocol reference documentation.

Places and scopes

Since resources must be closed at some point, they must be kept somewhere safe:

Lexical scope: with-resources

Similar to clojure.core/with-open, with-resources binds resources and provides a life cycle scope. When the evaluated expression returns or raises an exception, the contained resources are closed.

(with-resources [db (open-db ...)         ;; initialize resources
                 h  (create-handler db)   ;; pass dependencies
                 s  (run-server handler)] ;; etc]
    ;; do stuff with resources
    )
;; resources are out of scope and cleaned up

Global scope: defresource

In REPL sessions, it's common to put resources in a global var. defresource defines a var that contains a resource. Defining a new resource with the same var name will close the previous resource before initializing the new resource. defresource also adds a shutdown hook.

;; assign db handler to `my-db`, clean up previous resource in `my-db`
(defresource my-db (open-db ...))
;; if JVM is shut down, `my-db` will be cleaned up
;; see `close-on-shutdown!`

JVM scope: close-on-shutdown!

The JVM does not allow us to handle JVM shutdown caused by Signals in a try .. finally block like with-resources uses internally. If we need a resource to be closed on JVM shutdown, close-on-shutdown! registers a shutdown handler for the resource.

Resource collections

It can be useful to keep multiple resources in a collection. The collection can then be treated as a single resource.

Dependent named resources: mk-system

A system is a collection of named, fixed resources, and can be created with mk-system. When the system is closed, the contained resources are closed in reverse order. This provides a terse form of dependency handling:

(mk-system [db (open-db ...)         ;; initialize resources
            h  (create-handler db)   ;; pass dependencies
            s  (run-server handler)]
   ;; return map of sub-resources to use as the system
   {:db      db
    :handler h
    :server  s})

If no body form is provided, a map is created automatically:

(mk-system [db (open-db ...)         ;; initialize resources
            h  (create-handler db)   ;; pass dependencies
            s  (run-server handler)])
;; => map of sub-resources: {:db .. :h .. s ...}

Exceptions thrown (also during system initialization) are handled correctly; any initialized resources are closed in reverse order of initialization.

Systems are resources, so systems need to be placed somewhere.