toucan2.tools.hydrate — io.github.camsaul/toucan2 1.0.560

hydrate adds one or more keys to an instance or instances using various hydration strategies, usually using one of the existing keys in those instances. A typical use case would be to take a sequence of orders and add :user keys to them based on their values of the foreign key :user-id.

hydrate is how you use the hydration facilities; everything else in this namespace is only for extending hydration to support your models.

Toucan 2 ships with several hydration strategies out of the box:

Automagic Batched Hydration (via model-for-automagic-hydration)

hydrate attempts to do a batched hydration where possible. If the key being hydrated is defined as one of some table's model-for-automagic-hydration, hydrate will do a batched toucan2.select/select if a corresponding key (by default, the same key suffixed by -id) is found in the objects being batch hydrated. The corresponding key can be customized by implementing fk-keys-for-automagic-hydration.

(hydrate [{:user_id 100}, {:user_id 101}] :user)

Since :user is a hydration key for :models/User, a single toucan2.select/select will used to fetch Users:

(db/select :models/User :id [:in #{100 101}])

The corresponding Users are then added under the key :user.

Function-Based Batched Hydration (via batched-hydrate methods)

If the key can't be hydrated auto-magically with the appropriate model-for-automagic-hydration, hydrate will attempt to do batched hydration if it can find a matching method for batched-hydrate. If a matching function is found, it is called with a collection of objects, e.g.

(m/defmethod hydrate/batched-hydrate [:default :fields]
  [_model _k instances]
  (let [id->fields (get-some-fields instances)]
    (for [instance instances]
      (assoc instance :fields (get id->fields (:id instance))))))

Simple Hydration (via simple-hydrate methods)

If the key is not eligible for batched hydration, hydrate will look for a matching simple-hydrate method. simple-hydrate is called with a single instance.

(m/defmethod simple-hydrate [:default :dashboard]
  [_model _k {:keys [dashboard-id], :as instance}]
  (assoc instance :dashboard (select/select-one :models/Dashboard :toucan/pk dashboard-id)))

Hydrating Multiple Keys

You can hydrate several keys at one time:

(hydrate {...} :a :b)
  -> {:a 1, :b 2}

Nested Hydration

You can do recursive hydration by listing keys inside a vector:

(hydrate {...} [:a :b])
  -> {:a {:b 1}}

The first key in a vector will be hydrated normally, and any subsequent keys will be hydrated inside the corresponding values for that key.

(hydrate {...}
         [:a [:b :c] :e])
  -> {:a {:b {:c 1} :e 2}}

Forcing Hydration

Normally, hydration is skipped if an instance already has a non-nil value for the key being hydrated, but you can override this behavior by implementing needs-hydration?.

Flowchart

If you're digging in to the details, this is a flowchart of how hydration works:

                    hydrate ◄─────────────┐
                       │                  │
                       ▼                  │
                 hydrate-forms            │
                       │                  │
                       ▼                  │ (recursively)
                hydrate-one-form          │
                       │                  │
            keyword? ◄─┴─► sequence?      │
               │               │          │
               ▼               ▼          │
          hydrate-key    hydrate-key-seq ─┘
               │
               ▼
      (for each strategy) ◄────────┐
       ::automagic-batched         │
       ::multimethod-batched       │
       ::multimethod-simple        │
               │                   │ (try next strategy)
               ▼                   │
      can-hydrate-with-strategy?   │
               │                   │
        yes ◄──┴──► no ────────────┘
         │
         ▼
hydrate-with-strategy

*error-on-unknown-key* clj

batched-hydrate clj

(batched-hydrate model₁ k₂ instances)

Hydrate the key k in one or more instances of model. Implement this method to support batched hydration for the key k. Example implementation:

;; This method defines a batched hydration strategy for the :bird-type key for all models.
(m/defmethod hydrate/batched-hydrate [:default :is-bird?]
  [_model _k instances]
  ;; fetch a set of all the non-nil bird IDs in instances.
  (let [bird-ids           (into #{} (comp (map :bird-id) (filter some?)) instances)
        ;; if bird-ids is non-empty, fetch a map of bird ID => bird type
        bird-id->bird-type (when (seq bird-ids)
                             (select/select-pk->fn :bird-type :models/bird :id [:in bird-ids]))]
    ;; for each instance add a :bird-type key.
    (for [instance instances]
      (assoc instance :bird-type (get bird-id->bird-type (:bird-id instance))))))

Batched hydration implementations should try to be efficient, e.g. minimizing the number of database calls made rather than doing one call per instance. If you just want to hydrate each instance independently, implement simple-hydrate instead. If you are hydrating entire instances of some other model, consider setting up automagic batched hydration using model-for-automagic-hydration and possibly fk-keys-for-automagic-hydration.

can-hydrate-with-strategy? clj

(can-hydrate-with-strategy? model₁ strategy₂ k₃)

Can we hydrate the key k in instances of model using a specific hydration strategy?

Normally you should never need to call this yourself. The only reason you would implement it is if you are implementing a custom hydration strategy.

can-hydrate-with-strategy? is defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:164).

It caches methods using a methodical.impl.cache.watching.WatchingCache.

It uses the method combination methodical.impl.combo.threaded.ThreadingMethodCombination with the threading strategy :thread-last.

It uses the dispatcher methodical.impl.dispatcher.multi_default.MultiDefaultDispatcher with hierarchy #'clojure.core/global-hierarchy and prefs {}.

The default value is :default.

It uses the method table methodical.impl.method_table.standard.StandardMethodTable.

These primary methods are known:

• [:default :toucan2.tools.hydrate/automagic-batched :default], defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:252)

• [:default :toucan2.tools.hydrate/multimethod-batched :default], defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:419)

• [:default :toucan2.tools.hydrate/multimethod-simple :default], defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:473)

fk-keys-for-automagic-hydration clj

(fk-keys-for-automagic-hydration original-model₁ dest-key₂ hydrating-model₃)

The keys in we should use when automagically hydrating the key dest-key in instances of original-model with instances hydrating-model.

;; when hydrating :user in an order with a user, fetch the user based on the value of the :user-id key
;;
;; This means we take the value of :user-id from our order and then fetch the user with the matching primary key
;; (by default, :id)
(fk-keys-for-automagic-hydration :models/orders :user :models/user) => [:user-id]

The model that we are hydrating with (e.g. :models/user) is determined by model-for-automagic-hydration.

By default fk-keys-for-automagic-hydration is just the key we're hydrating, with -id appended to it; e.g. if we're hydrating :user the default FK key to use is :user-id. If this convention is fine, you do not need to implement this method. If you want to do something that does not follow this convention, like hydrate a :user key based on values of :creator-id, you should implement this method.

Example implementation:

;; hydrate orders :user key using values of :creator-id
(m/defmethod hydrate/fk-keys-for-automagic-hydration [:model/orders :user :default]
  [_original-model _dest-key _hydrating-model]
  [:creator-id])

Tips

When implementing this method, you probably do not want to specialize on the hydrating-model (the third part of the dispatch value) -- the model to use is normally determined by model-for-automagic-hydration. The only time you might want to specialize on hydrating-model is you wanted to do something like

;; by default when we hydrate the :user key with a :models/user, use :creator-id as the source FK
(m/defmethod hydrate/fk-keys-for-automagic-hydration [:default :model/orders :user :models/user]
  [_original-model _dest-key _hydrating-model]
  [:creator-id])

fk-keys-for-automagic-hydration is defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:256).

It caches methods using a methodical.impl.cache.watching.WatchingCache.

It uses the method combination methodical.impl.combo.threaded.ThreadingMethodCombination with the threading strategy :thread-last.

It uses the dispatcher methodical.impl.dispatcher.multi_default.MultiDefaultDispatcher with hierarchy #'clojure.core/global-hierarchy and prefs {}.

The default value is :default.

It uses the method table methodical.impl.method_table.standard.StandardMethodTable.

These primary methods are known:

• :default, defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:301)

These aux methods are known:

:around methods:

• :default, defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:307)

global-error-on-unknown-key clj

Whether hydration should error when it encounters a key that has no hydration methods associated with it (default: false). Global default value. You can bind *error-on-unknown-key* to temporarily override this value.

hydrate clj

(hydrate instance-or-instances)

(hydrate instance-or-instances & ks)

Hydrate the keys ks in a single instance or sequence of instances. See toucan2.tools.hydrate for more information.

hydrate-with-strategy clj

(hydrate-with-strategy model strategy₁ k instances)

Hydrate the key k in instances of model using a specific hydration strategy.

Normally you should not call this yourself. The only reason you would implement this method is if you are implementing a custom hydration strategy.

hydrate-with-strategy is defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:174).

It caches methods using a methodical.impl.cache.watching.WatchingCache.

It uses the method combination methodical.impl.combo.threaded.ThreadingMethodCombination with the threading strategy :thread-last.

It uses the dispatcher methodical.impl.dispatcher.multi_default.MultiDefaultDispatcher with hierarchy #'clojure.core/global-hierarchy and prefs {}.

The default value is :default.

It uses the method table methodical.impl.method_table.standard.StandardMethodTable.

These primary methods are known:

• :toucan2.tools.hydrate/automagic-batched, defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:376)

• :toucan2.tools.hydrate/multimethod-batched, defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:445)

• :toucan2.tools.hydrate/multimethod-simple, defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:477)

model-for-automagic-hydration clj

(model-for-automagic-hydration original-model₁ k₂)

The model that should be used to automagically hydrate the key k in instances of original-model.

(model-for-automagic-hydration :some-table :user) :-> :myapp.models/user

Dispatches off of the [[toucan2.protocols/dispatch-value]] (normally [[toucan2.protocols/model]]) of the instance being hydrated and the key k that we are attempting to hydrate. To hydrate the key k for any model, you can use :default in your defmethod dispatch value. Example implementation:

;; when hydrating the :user key for *any* model, hydrate with instances of the model :models/user
;;
;; By default, this will look for values of :user-id in the instances being hydrated and then fetch the instances of
;; :models/user with a matching :id
(m/defmethod hydrate/model-for-automagic-hydration [:default :user]
  [_original-model _k]
  :models/user)

;; when hydrating the :user key for instances of :models/orders, hydrate with instances of :models/user
(m/defmethod hydrate/model-for-automagic-hydration [:models/orders :user]
  [_original-model _k]
  :models/user)

Automagic hydration looks for the fk-keys-for-automagic-hydration in the instance being hydrated, and if they're all non-nil, fetches instances of model-for-automagic-hydration with the corresponding toucan2.model/primary-keys. Thus you might also want to implement

• fk-keys-for-automagic-hydration for the model whose instances hydrating and the key being hydrated (e.g. :models/orders and :user)

• toucan2.model/primary-keys for the model you want to hydrate with (e.g. :models/user)

Tips

• You probably don't want to write an implementation for [<some-model> :default] or [:default :default], unless you want every key that we attempt to hydrate to be hydrated by that method.

model-for-automagic-hydration is defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:204).

It caches methods using a methodical.impl.cache.watching.WatchingCache.

It uses the method combination methodical.impl.combo.threaded.ThreadingMethodCombination with the threading strategy :thread-last.

It uses the dispatcher methodical.impl.dispatcher.multi_default.MultiDefaultDispatcher with hierarchy #'clojure.core/global-hierarchy and prefs {}.

The default value is :default.

It uses the method table methodical.impl.method_table.standard.StandardMethodTable.

These primary methods are known:

• :default, defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:248)

needs-hydration? clj

(needs-hydration? model₁ k₂ instance)

Whether an instance of model needs the key k to be hydrated. By default, this is true if (get instance k) is nil. You can override this if you want to force a key to always be re-hydrated even if it is already present.

needs-hydration? is defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:185).

It caches methods using a methodical.impl.cache.watching.WatchingCache.

It uses the method combination methodical.impl.combo.threaded.ThreadingMethodCombination with the threading strategy :thread-last.

It uses the dispatcher methodical.impl.dispatcher.multi_default.MultiDefaultDispatcher with hierarchy #'clojure.core/global-hierarchy and prefs {}.

The default value is :default.

It uses the method table methodical.impl.method_table.standard.StandardMethodTable.

These primary methods are known:

• :default, defined in toucan2.tools.hydrate (toucan2/tools/hydrate.clj:193)

simple-hydrate clj

(simple-hydrate model₁ k₂ instance)

Implementations should return a version of map instance with the key k added.