Liking cljdoc? Tell your friends :D

Asami Database Plugin for Fulcro RAD

This is a plugin for Fulcro RAD that adds support for using Asami databases (both in-memory and disk-based ones) as the back-end technology.

Status

Stable

Maintenance mode

The plugin is now stable and in maintenance mode. Feel free to report bugs and submit PRs, they will be reviewed and processed.

Usage

Extra requirements

Minimal required version of Clojure is 1.11.0, ClojureScript 1.10.866 (due to reliance on update-vals).

Every entity stored in Asami must have the property :id <ident>. Entities created via a RAD form get this automatically.

Add the dependency

;; in deps.edn:
{:deps {cz.holyjak/fulcro-rad-asami {:mvn/version "RELEASE"}
        ;; pick ONE of Pathom v2 and Pathom3:
        com.wsscode/pathom3 {:mvn/version "2023.01.31-alpha"}
        ;com.wsscode/pathom {:mvn/version "2.4.0"}
}}

Usage example code (Pathom 3)

(ns com.example.components.parser ; Pathom 3
  (:require
    [com.example.model :as your-model]
    [com.fulcrologic.rad.attributes :as attr]
    [com.fulcrologic.rad.form :as form]
    [com.fulcrologic.rad.middleware.save-middleware :as r.s.middleware]
    [com.fulcrologic.rad.pathom :as pathom]
    [com.fulcrologic.rad.resolvers :as res]
    [cz.holyjak.rad.database-adapters.asami :as asami]
    [cz.holyjak.rad.database-adapters.asami-options :as aso]))

(def config {::asami/databases {:production {:asami/driver :local #_:mem, :asami/database "fulcro-rad-demo"}}})

;; config should contain the key ::aso/databases, see start-connections docstring
(def asami-connections (asami/start-connections config))

(def automatic-resolvers
  (vec
    (concat
      (res/generate-resolvers your-model/all-attributes)
      (asami/generate-resolvers your-model/all-attributes :production))))

(def save-middleware (r.s.middleware/wrap-rewrite-values (asami/wrap-save))) ; the wrap-rewrite-values is optional

;; See examples of this in cz.holyjak.rad.database-adapters.asami-spec
(def parser
  (let [env-middleware (-> (attr/wrap-env all-attributes)
                           (form/wrap-env save/middleware delete/middleware)
                           (asami/wrap-env (fn [env] asami-connections))
                           (blob/wrap-env bs/temporary-blob-store {:files         bs/file-blob-store
                                                                   :avatar-images bs/image-blob-store}))]
    (pathom3/new-processor {#_"your config here..."} env-middleware []
      [automatic-resolvers your-model/custom-resolvers])))

Usage example code (Pathom 2)

(ns com.example.components.parser ; Pathom 2
  (:require
    [com.example.model :as your-model]
    [com.fulcrologic.rad.attributes :as attr]
    [com.fulcrologic.rad.form :as form]
    [com.fulcrologic.rad.middleware.save-middleware :as r.s.middleware]
    [com.fulcrologic.rad.pathom :as pathom]
    [com.fulcrologic.rad.resolvers :as res]
    [cz.holyjak.rad.database-adapters.asami :as asami]
    [cz.holyjak.rad.database-adapters.asami-options :as aso]
    [cz.holyjak.rad.database-adapters.asami.pathom :as asami.pathom))

(def config {::asami/databases {:production {:asami/driver :local #_:mem, :asami/database "fulcro-rad-demo"}}})

;; config should contain the key ::aso/databases, see start-connections docstring
(def asami-connections (asami/start-connections config))

(def automatic-resolvers
  (vec
    (concat
      (res/generate-resolvers your-model/all-attributes)
      (asami/generate-resolvers your-model/all-attributes :production))))

(def save-middleware (r.s.middleware/wrap-rewrite-values (asami/wrap-save))) ; the wrap-rewrite-values is optional

;; See examples of this in cz.holyjak.rad.database-adapters.asami-spec
(def parser
    (pathom/new-parser {#_"your config here..."}
        [(attr/pathom-plugin all-attributes)
         (form/pathom-plugin save-middleware (asami/wrap-delete))
         (asami.pathom/pathom-plugin (fn [env] asami-connections))]
        [automatic-resolvers your-model/custom-resolvers]))

See a complete usage example in https://github.com/fulcrologic/fulcro-rad-demo .

It is better to call asami.core/shutdown when shutting down the backend to ensure that the database files are not unnecessarily large. Though you can likely simply rely on the JVM shutdown hook calling this, which Asami itself registers. (They are created with more space so that data can be inserted quickly and shutdown trims them to the actual content.) (Beware: as of Asami 2.3.2, when you call shutdown, it will close files but not reset the internal connections map and subsequent asami/start-connections will not re-initialize them properly. So avoid calling shutdown repeatedly during REPLing.)

Configuration

rad-asami specific configuration options

You can set the following on an id attribute (i.e. one with ao/identity? true and a ao/schema; ::asami here is cz.holyjak.rad.database-adapters.asami):

::asami/no-batch? - by default, all generated id resolvers are batched. In some rare cases you might want to disable that - so set no-batch? to true. (Note: Since Asami runs in-memory, batching most likely doesn’t really matter.)
::asami/owned-entity? marks the entity as dependant, i.e. it can only exist as part of another entity - f.ex. an OrderLine can only exist as a part of an Order. It is marked in Asami as "owned" by the parent entity and thus (d/entity parent-entity) will include its full data and, most importantly, removing it from the :ref attribute on the parent will delete it fully from the DB (notice that this "cascading delete" is a feature of this adapter, not of Asami itself).

::asami/wrap-resolve - a function wrapping the resolver’s resolve function. Example:

::asami/wrap-resolve
(fn [resolve-fn]
     (fn [pathom-env input]
       (println "Running resolve for input" input)
       (resolve-fn pathom-env input)))

Tips

Warning: beware inserting nested entities

The generated id resolvers use d/entity to fetch the data. That has the effect of pulling the entity and all nested entities. Normally that is not a problem when you only insert data via save-form etc., because this will break any data into quadruplets and insert even nested entities as top entities. But if you insert data not as quadruplets but as an entity tree as here:

@(d/transact *conn* {:tx-data [{:id [::person/id "ann"]
                                     ::person/id "ann"
                                     ::person/addresses [{:id [::address/id "a-one"] ; <- nested entity!
                                                          ::address/id "a-one"
                                                          ::address/street "First St."}
                                                         {:id [::address/id "a-two"]
                                                          ::address/id "a-two"
                                                          ::address/street "Second St."}]}]})

then addresses will become nested entities and d/entity will return person together with the whole value. (Notice that setting nested? to false on d/entity has no effect here - this option only makes sense with the value true for references to other top entities that you want to pulled whole).

(Notice you can still fetch an address separately with (d/entity conn [::address/id "a-one"]), thanks to having set that :id.)

WIP A problem with pulling nested entities is that Pathom 3 v.2022.10.19-alpha apparently throws away this nested data. I’m currently looking into this

To create multiple top-level entities using the entity tx form, this might work (I have not tested it properly):

(d/transact conn {:tx-data [{:id "a-one"
                             :address/id     "a-one"
                             :address/street "First St."}
                            {:id               [:person/id "ann"]
                             :person/id        "ann"
                             :person/addresses [{:id "a-one"}]}]})

To create multiple top-level entities using the entity tx form, this normally works:

(d/transact conn {:tx-data [{:id [:address/id "a-one"]
                             :address/id     "a-one"
                             :address/street "First St."}
                            {:id               [:person/id "ann"]
                             :person/id        "ann"
                             :person/addresses [{:id [:address/id "a-one"]}]}]})

Lookup refs

When inserting data manually, remember to set :id <ident>. You can then use it as a lookup ref, e.g. in add: [:db/add [:id <ident>] <prop> <val>].

Utilities for generating transactions

Use functions such as write/retract-entity-txn and write/delta→txn-map-with-retractions if you want to make transactions to delete or update entities in a way consistent with RAD-managed entities.

Wrap an auto-generated resolver

You can provide a function that is invoked around an autogenerated resolver for an entity by setting ::asami/wrap-resolve on the ID attribute. Notice that id resolvers typically produce a vector because they are batched.

Example 1. Wrapping an auto-generated resolver

(defattr id :order/id :uuid
  {ao/identity? true
   ao/schema :production
   :cz.holyjak.rad.database-adapters.asami/wrap-resolve
   (fn wrap-resolve [res]
     (fn decorated-resolve [env in]
       (println "order-id resolver in=" in)
       (doto (res env in)
         (->> (println "order-id resolver output=")))))})

Troubleshooting

You can find many great troubleshooting tips in my Pathom3 + Fulcro Lab Notes.

Enable debug logging

You can enable debug logging for the adapter. With fulcro-rad-demo or fulcro-template you can configure this in e.g. its dev.edn:

- {:taoensso.timbre/logging-config {:min-level :info}}
+ {:taoensso.timbre/logging-config {:min-level [[#{"cz.holyjak.rad.database-adapters.asami.*"} :debug]
+                                               [#{"*"} :info]]}}

Exploring the data

Fetch all the entity-attribute-value triples from the database:

(d/q [:find '?e '?a '?v :where '[?e ?a ?v]]
       (d/db (:production asami-connections)))

More info

Important characteristics of Asami and the adapter

The order of multi-valued attributes is lost (Asami returns them as sets, which we turn into a vector).

As of Asami 2.3.2 you cannot create an entity and refer to the entity from another one in the same transaction when using the entity form of tx-data. If the entity and reference are both created using the quadruplets form ([:db/add <entity> <attr> <val>) then this works.

Implementation details

We assoc to each persistent entity :id <ident> (see Asami’s Identity Values) so that we can easily refer to it in statements and from other entities. This is then used as a lookup ref in insert/update statements and in :ref attributes of other properties. (:ref attributes stored via a form are automatically translated into this form.) However this property is dissoc-ed when reading. (We could likely also use :db/ident instead though this has not been tested.)

We store the full ident in the :id because we cannot be sure that the ID values are globally unique though we know that Fulcro would break if they were not unique for the given entity. (Actually they should be globally unique, being UUIDs, but we might want to support other kinds of IDs in the future that do not guarantee this. We could store the full ident only on such attributes - and maybe we will.)

Design decisions

Quadruples over entities We translate each Fulcro entity diff into a series of quadruplet assertions and retractions and transact these. The reason for this is that we might want to transact multiple new entities that refer to each other in a single transaction (think of saving a form with a subform). I am not sure Asami tempids work for this and in any case they are not ideal because, in the face of no schema, they are just negative integers and then even regular attribute value that happen to be negative integers matching one of the tempids would be replaced with a reference. Instead, we use lookup ids such as {:id [:entity/id #uuid "some-value"]} but these require that the entity already exists, when used in the entity form, while quadruplets manage to create a new entity and resolve references to it (see https://github.com/quoll/asami/pull/2). One of the disadvantages is that we cannot use the attribute' or attribute+ shorthand forms.

Limitations

Limitations and features that are not supported:

Currently, IDs must be of the type uuid and new entities need this set to a tempid so that the entity is created before being referred to and with correct attributes
RAD Datomic-like native IDs are not supported yet (see parts of code marked with FEAT-NAT-IDS (incomplete), should we ever implement this)
ao/identities must have exactly one element

Not tested:

Multiple databases / schemas

Development

Testing

Run tests: clj -M:pathom3:test:run-tests

Also see the (comment ..) at the bottom of most -spec tests for running those in the REPL.

Example 2. Focusing a test

(specification "descr." :focus ...)

then run (fulcro-spec.reporters.repl/run-tests (comp :focus meta))

Releasing

First, update version in build.clj:

- (def version "1.0.2")
+ (def version "1.0.3")

and ideally also scm > tag in pom.xml,

then build and release to Clojars:

bb run # build
clj -Spom # refresh pom
env CLOJARS_USERNAME=holyjak CLOJARS_PASSWORD=<secret> clj -X:deploy

Can you improve this documentation? These fine people already did:
Jakub Holy & Jakub HolýEdit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

Keyboard shortcuts Report a problem cljdoc on GitHub

× close