Liking cljdoc? Tell your friends :D

Application system management

Let's start an application:

(let [config (get-config)
      db (init-db {:config config})
      es (init-es {:config config})
      app (init-app {:config config :db db :es es})]
  app)

;; or using a sys map:

(let [sys {:config (get-config)}
      sys (assoc sys :db (init-db sys))
      sys (assoc sys :es (init-es sys))
      sys (assoc sys :app (init-app sys))]
  app)

;; With a little helper function we can thread the system through:

(defn assoc-with [sys id f]
  (assoc sys id (f sys)))

(-> {}
    (assoc-with :config init-config)
    (assoc-with :db init-db)
    (assoc-with :es init-es)
    (assoc-with :app init-app))

;; Let's make the dependencies explicit:
 
(defn assoc-with [sys id f deps]
  (assoc sys id (f (select-keys sys deps))))

(-> {}
    (assoc-with :config init-config [])
    (assoc-with :db init-db [:config])
    (assoc-with :es init-es [:config])
    (assoc-with :app init-app [:config :db :es]))

;; Using topological sort to do the work of ordering lets us define our system declaratively:

[{:id :config :init init-config :deps []}
 {:id :db :init init-db :deps [:config]}
 {:id :es :init init-config :deps [:config]}
 {:id :app :init init-config :deps [:db :es]}]

;; or as map:

{:config {:init init-config :deps []}
 :db {:init init-db :deps [:config]}
 :es {:init init-config :deps [:config]}
 :app {:init init-config :deps [:db :es]}}
 
;; This is all that's really needed for production, and basically just sugar, 
;; the first version above works just as well.

;; However in development or when shutting down a service or job we'd like to
;; 'stop' a component as well (for purposes of reloading bindings in the repl,
;; when component 'restarts' it'll pick up the new bindings) so we should
;; further annotate our 'components' with a stop fn:

{:config {:init init-config :deps []}
 :db {:init init-db :deps [:config] :stop stop-db}
 :es {:init init-config :deps [:config]}
 :app {:init init-config :deps [:db :es] :stop stop-app}}
 

To keep the 'system' in a sane state when we 'stop' a component its dependencies should be stopped recursively similarly to how they are started, but dependencies should be stopped after cmp itself is stopped. When starting a components dependencies are started before cmp itself is started.

With some function juggling we can create pure starter and stopper functions that take the old system map, the full set of components and the id of the component to start/stop and return a new system with the cmp started (and assoced in) or stopped (and dissoced from) the old sys map, recursively.

For repl reload work flow a sys var (using alter-var-root) can be updated using these starter/stoppper functions which is especially useful in development. The concept and way of working is very similar to the one for https://github.com/stuartsierra/component but using maps and pure functions instead of records and protocols to manage components and lifecycles.

The core functionality is in realm.core with an implementation in realm.impl.system. A component's start function can return any value but nil (which would indicate the component isn't running). A component's stop function's return value is ignored.

Example system:

(require '[realm.impl.system :as sys])
(require '[realm.impl.validate :as validate])

 (def sys {})

 (def cmps {:cmp-1 {:init 1}
            :cmp-2 {:value 2}
            :cmp-3 {:init (fn [cmp-2]
                            cmp-2)
                    :deps :cmp-2}
            :cmp-4 {:deps   [:cmp-1 :cmp-3]
                    :rename {:cmp-1 :first-cmp}
                    :setup   (fn [{:keys [first-cmp cmp-3] :as sys}]
                               sys)
                    :done   (fn [this]
                                 (tap> {:cmp-3-done this}))}})

  (validate/validate-cmps cmps)
  (sys/start #'sys cmps :cmp-4) ;;updates my-sys var
  (sys/stop #'sys cmps :cmp-4)  ;;updates my-sys var and taps :cmp-3-done
  (tap> sys)

The sys var is thread local so this won't work if for some reason you want to update the sys from multiple threads. In that case using an agent should work:

(defonce sys (agent {}))

;;TODO: add interface fns to 'send' the sys starter/stopper fn to the agent, this will
garantuee that the sys will only ever be updated by one updater at a time. Once
a cmp is started or stopped it can't be started or stopped again however this
should never happen in parallel in more than one thread at a time!

Realm doesn't tie component data maps to any particular component implementation. One way to do this is to init a component with a record that implements a protocol:

(defprotocol Bar
  (bar [this arg]))

(defrecord Foo [es db]
  Bar
  (bar [this arg]
    (+ 100 arg)))
  
 (def cmp {:init (fn [sys] (map->Foo ctx))}) 

Or withouth using protocols and records:

;; Implementation fn
(defn bar_ [sys arg]
  (+ 100 arg))

;; Definition of component in implementation namespace
(def cmp  {:init (fn [sys]
                   {:impl {:bar bar_}
                    :sys ctx})})

;; Generic interface fn:
(defn bar [{:keys [impl sys]} arg]
  ((:bar impl) sys arg))


;;Use initialized component
(let [sys ((:init cmp) {:es 1 :db 2})]
  (bar sys 100))

What we want/need from a state management solution:

  • easy unit/integration testing
  • easy repl reload flow
  • easy swapping in of dev/test/sit/staging/production components
  • use of env/sys first parameter convention
  • any system components can/will have uses/using/deps, begin/open/init/create/start/setup and end/close/release/teardown/stop/halt interface fns
  • run multiple systems at the same time
  • usable error reporting when starting/stopping fails
  • functionality to recover from failed starts

Similar libraries

Can you improve this documentation?Edit on Codeberg

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close