Readme — org.rssys/context 0.1.4

Intro

Context is a library designed to manage system state.

Motivation

System state (or context) should be in one place. Context is like a central database holding the current world. Every system component should be able to get any information about the system state from one place in a consistent manner.
State management should be as simple as possible, avoiding framework-style constraints for application code. Ideally, context should be a pure Clojure atom produced from a declarative data structure on the fly.
State management should be flexible to facilitate the building of multi-tenant systems. Context should not be hard linked to a specific namespace. Ideally, context is a parameter for any system component which performs some business logic.
System state should have declarative information structure which allows one to read and understand the current state without the need for special tools. What is the whole system config? What parameters were used to start a database? What is the current implementation of a cache component? Which components are started? All of these questions should have the immediate answer from context.
Dependencies between system components should be resolved accurately.

For a long time I used the mount library and it is a very good solution for state management. But now, for my projects, I need to see the system state in one place. Other Clojure solutions force me to arrange my components in a specific manner (e.g. using defrecords and protocols). These framework-style constraints sometimes hamper the building of flexible systems by requiring that all components of an application follow the same pattern.

Idea of the Context

Note, that system behaviour (I/O) depends only on context. There is no special places for state inside the System.

Demo App

See web app example

Usage

Add to :deps section of deps.edn the context dependency (see latest version above):

:deps { org.rssys/context {:mvn/version "0.1.4"}}

Require necessary namespaces:

(require '[org.rssys.context.core :as ctx])
(require '[unifier.response :as r])

Minimal component

Each stateful object, e.g., a database connection, a cache, etc., in a system is called a component. The minimal component is a map with two required parameters: :id (keyword) and :config (map or function). The minimal component can be registered in a system context but it cannot be started (it is not executable yet).

;; define context with a minimal component
@(let [*ctx (atom {})
       system [{:id :web :config {}}]]
   (ctx/build-context *ctx system))
;; => #:context{:components {:web {:id :web, :config {}, :state-obj nil, :status :stopped, :start-deps []}}}

The context

The context is a pure Clojure atom containing a map of components {:id-1 component-state1, …}. All components are placed under the :context/components key. This key can be overridden using the *components-path-vec* variable.

(binding [ctx/*components-path-vec* []]
  @(let [*ctx (atom {})
         system [{:id :web :config {}}]]
     (ctx/build-context *ctx system)))
;;=> {:web {:id :web, :config {}, :state-obj nil, :status :stopped, :start-deps []}}

The context can be built in a step-by-step manner:

(def *ctx (atom {}))
;;=> #'user/*ctx

;; create a minimal component
(ctx/create! *ctx {:id :web :config {}})
;;=> #unifier.response.UnifiedSuccess{:type :unifier.response/created, :data :web, :meta nil}

;; create another one
(ctx/create! *ctx {:id :db :config {:host "localhost" :port 1234}})
;;=> #unifier.response.UnifiedSuccess{:type :unifier.response/created, :data :db, :meta nil}

@*ctx
;;=>
#:context{:components {:web {:id :web, :config {}, :state-obj nil, :status :stopped, :start-deps []},
                       :db {:id :db,
                            :config {:host "localhost", :port 1234},
                            :state-obj nil,
                            :status :stopped,
                            :start-deps []}}}

Response codes

Context uses the Unified responses library. Every successful operation with a context has type unifier.response.UnifiedSuccess and every failure has type unifier.response.UnifiedError.

In this example, an attempt to create a component with a pre-existing id returns a failure code:

(ctx/create! *ctx {:id :db :config {:host "localhost" :port 1234}})
;;=>
#unifier.response.UnifiedError{:type :unifier.response/conflict,
                               :data :db,
                               :meta "component with such id is already exist"}

The r/success? and r/error? functions from the Unified responses library can be used to categorize the result of a context operation.

Start / stop the component

To start the component it has to be executable. The executable component has defined start/stop functions. The start function (supplied via the :start-fn key) takes one argument (config value) and should return a stateful object. The stop function (under :stop-fn) takes one argument (a stateful object) and should close connections, release resources and so on.

(def *ctx (atom {}))
;; => #'user/*ctx

;; a minimal executable component. Start function has a :config value as argument.
(ctx/create! *ctx {:id :web :config {} :start-fn (fn [config]) :stop-fn (fn [obj-state])})
;; => #unifier.response.UnifiedSuccess{:type :unifier.response/created, :data :web, :meta nil}

(ctx/start! *ctx :web)
;; => #unifier.response.UnifiedSuccess{:type :unifier.response/success, :data :web, :meta nil}

(ctx/stop! *ctx :web)
;; => #unifier.response.UnifiedSuccess{:type :unifier.response/success, :data :web, :meta nil}

After stop a component state is set to a value EmptyState.

Component dependency management

Context library has dependency management. Every component dependency should be declared in the :start-deps collection: vector [] or set #{}, using other component id’s. That is, the :start-deps collection of component C contains the id’s of the components that should be started before C.

Example: If :web component depends on :cache component, and :cache component depends on :db component, then it can be declared like this:

(def *ctx (atom {}))
;; => #'user/*ctx

(ctx/create! *ctx {:id :db :config {} :start-deps [] :start-fn (fn [config]) :stop-fn (fn [obj-state])})
;; => #unifier.response.UnifiedSuccess{:type :unifier.response/created, :data :db, :meta nil}

(ctx/create! *ctx {:id :cache :config {} :start-deps [:db] :start-fn (fn [config]) :stop-fn (fn [obj-state])})
;; => #unifier.response.UnifiedSuccess{:type :unifier.response/created, :data :cache, :meta nil}

(ctx/create! *ctx {:id :web :config {} :start-deps [:cache] :start-fn (fn [config]) :stop-fn (fn [obj-state])})
;; => #unifier.response.UnifiedSuccess{:type :unifier.response/created, :data :web, :meta nil}

;; the start of the :web component causes the start of :db and :cache components, respectively.
(ctx/start! *ctx :web)
;; => #unifier.response.UnifiedSuccess{:type :unifier.response/success, :data :web, :meta nil}

;; check which components are started
(ctx/started-ids *ctx)
;; => [:db :cache :web]

The cyclic dependency check between components is implemented. To control the behavior of cyclic dependency check use flag *ignore-cyclic-deps?*. If the flag is false (default) then an Exception will be thrown if cyclic dependency is detected. If the flag is true (you know what you are doing!), then the cyclic dependency loop will be ignored and context will be forced to start the components.

Feature toggling

The system may have components with the status :disabled. During the system start any disabled components will be ignored (start/stop functions will not run). To disable a component just add to components' config map the key :context/component-disabled and true as a value (or function that prepares components' config should return map with key-value pair {:context/component-disabled true}. If the component has disabled dependencies, then it will be automatically disabled too during the system start. The key :context/component-disabled can be overridden by a dynamic var *component-disabled*.

Minimal system example

(let [*ctx (atom {})
      system-map [
                  {:id         :cfg                     ;; cfg component will prepare config for all context
                   :config     {}
                   :start-deps []
                   :start-fn   (fn [config]
                                 (println "reading config data from OS & JVM environment variables or config file")
                                 {:db    {:host "localhost" :port 1234 :user "sa" :password "*****"}
                                  :cache {:host "127.0.0.1" :user "cache-user" :pwd "***"}
                                  :web   {:host "localhost" :port 8080 :root-context "/main"}})
                   :stop-fn    (fn [obj-state])}

                  {:id         :db
                   :config     (fn [ctx] (-> (ctx/get-component-value ctx :cfg) :state-obj :db))
                   :start-deps [:cfg]
                   :start-fn   (fn [config] (println "starting db" :config config))
                   :stop-fn    (fn [obj-state] (println "stopping db..."))}

                  {:id         :cache
                   :config     (fn [ctx] (-> (ctx/get-component-value ctx :cfg) :state-obj :cache))
                   :start-deps [:cfg :db]
                   :start-fn   (fn [config] (println "starting cache" :config config))
                   :stop-fn    (fn [obj-state] (println "stopping cache..."))}

                  {:id         :log
                   :config     {:output "stdout"}
                   :start-deps []
                   :start-fn   (fn [config] (println "starting logging" :config config))
                   :stop-fn    (fn [obj-state] (println "stopping logging..."))}

                  {:id         :web
                   :config     (fn [ctx] (-> (ctx/get-component-value ctx :cfg) :state-obj :web))
                   :start-deps [:cfg :db :cache :log]
                   :start-fn   (fn [config]
                                 (println "starting web" :config config)
                                 (println "pass the whole context as atom to web handler:" *ctx))
                   :stop-fn    (fn [obj-state] (println "stopping web..."))}
                  ]
      ]
  (ctx/build-context *ctx system-map)
  (println "list of all registered components:" (ctx/list-all-ids *ctx))
  (ctx/start-all *ctx)
  (do
    (println "do some business logic or control functions with context"))
  (println  "list of all started components:"  (ctx/started-ids *ctx))
  (ctx/stop-all *ctx))

list of all registered components: [:cfg :db :cache :log :web]
reading config data from OS & JVM environment variables or config file
starting db :config {:host localhost, :port 1234, :user sa, :password *****}
starting cache :config {:host 127.0.0.1, :user cache-user, :pwd ***}
starting logging :config {:output stdout}
starting web :config {:host localhost, :port 8080, :root-context /main}
pass the whole context as atom to web handler: #object[clojure.lang.Atom 0x25f7aecb ...
do some business logic or control functions with context
list of all started components: [:cfg :db :cache :log :web]
stopping web...
stopping cache...
stopping db...
stopping logging...

;; => #unifier.response.UnifiedSuccess{:type :unifier.response/success, :data [:cfg :db :cache :web :log], :meta nil}

Component’s anatomy

Complete structure of component:

{:id :db,                 ;; component identifier
 :config {},              ;; config is a map or fn with one arg - current whole context value
 :start-deps [],          ;; dependencies which should be started before this component. Use set #{} or vector []..
 :start-fn #object[fn],   ;; fn which starts this component with one argument (:config value)
 :stop-fn #object[fn],    ;; fn which stops this component with one argument (stateful object)
 :state-obj nil,          ;; stateful object (any value) or EmptyState if component is not run
 :status :started,        ;; component status
 :stop-deps [:cache]}     ;; dependencies which should be stopped before this component (managed by context)

CRUD-like functions

There are some useful low-level API functions for managing component state:

(def *ctx (atom {}))
(ctx/create! *ctx {:id :db :config {} })
(ctx/get-component *ctx :db)
(ctx/get-component-value @*ctx :db)
(ctx/update! *ctx {:id :db :config {:a 1 :b 2} :start-deps []})  ;; update the whole value
(ctx/set-config! *ctx :db {:a 42})            ;; modify :config value
(ctx/delete! *ctx :db)                        ;; if status is :started then it cannot be deleted

Other functions

(ctx/start-all *ctx)
(ctx/stop-all *ctx)
(ctx/start-some *ctx [:db :cache])
(ctx/stop-some *ctx [:db :cache])
(ctx/started-ids *ctx)          ;; get list of started components
(ctx/stopped-ids *ctx)          ;; get list of stopped components
(ctx/started? *ctx :db)
(ctx/stopped? *ctx :db)
(ctx/disabled? *ctx :db)         ;; check if component is disabled
(ctx/disabled-ids *ctx)          ;; get list of disabled components
(ctx/isolated-stop! *ctx :db)    ;; isolated stop the component ignoring all its dependencies
(ctx/isolated-start! *ctx :db)   ;; isolated start the component ignoring all its dependencies

Also see helpers ns (org.rssys.context.helpers).

(require '[org.rssys.context.helpers :as h])
(h/config-value ctx :db)  ;; returns :config value of :db. It doesn't matter ctx is atom or value.
(h/state-value ctx :db)  ;; returns :state-obj value of db. It doesn't matter ctx is atom or value.
(h/build-state-map ctx)  ;;  returns `{:component-id current-component-state}` map

h/state-value function is aware of clojure.lang.IDeref, so it is possible to use async value (future, promise) returned from start-fn. This may be helpful when you want to perform asynchronous connection to database and use it in another component.

h/build-state-map function is very useful in Ring handlers.

Building the project

To build a project run make <command>. List of available commands:

clean - clear target folder
javac - compile java sources
compile - compile clojure code
build - build jar file (as library)
install - install jar file (library) to local .m2
deploy - deploy jar file (library) to clojars.org
conflicts - show class conflicts (same name class in multiple jar files)
release - release artifact. To release artifact run clojure -A:pbuild release.
bump - bump version artifact in build file. E.g: clojure -A:pbuilder bump beta. Parameter should be one of: major, minor, patch, alpha, beta, rc, qualifier, release.