toucan2.tools.hydrate — io.github.camsaul/toucan2 1.0.0-beta2-SNAPSHOT

hydrate adds one or more keys to an instance or instances using various hydration strategies, usually using one of the existing keys in those instances. A typical use case would be to take a sequence of orders and add :user keys to them based on their values of the foreign key :user-id.

hydrate is how you use the hydration facilities; everything else in this namespace is only for extending hydration to support your models.

Toucan 2 ships with several hydration strategies out of the box:

Automagic Batched Hydration (via model-for-automagic-hydration)

hydrate attempts to do a batched hydration where possible. If the key being hydrated is defined as one of some table's model-for-automagic-hydration, hydrate will do a batched toucan2.select/select if a corresponding key (by default, the same key suffixed by -id) is found in the objects being batch hydrated. The corresponding key can be customized by implementing fk-keys-for-automagic-hydration.

(hydrate [{:user_id 100}, {:user_id 101}] :user)

Since :user is a hydration key for :models/User, a single toucan2.select/select will used to fetch Users:

(db/select :models/User :id [:in #{100 101}])

The corresponding Users are then added under the key :user.

Function-Based Batched Hydration (via batched-hydrate methods)

If the key can't be hydrated auto-magically with the appropriate model-for-automagic-hydration, hydrate will attempt to do batched hydration if it can find a matching method for batched-hydrate. If a matching function is found, it is called with a collection of objects, e.g.

(m/defmethod hydrate/batched-hydrate [:default :fields]
  [_model _k rows]
  (let [id->fields (get-some-fields rows)]
    (for [row rows]
      (assoc row :fields (get id->fields (:id row))))))

Simple Hydration (via simple-hydrate methods)

If the key is not eligible for batched hydration, hydrate will look for a matching simple-hydrate method. simple-hydrate is called with a single row.

(m/defmethod simple-hydrate [:default :dashboard]
  [_model _k {:keys [dashboard-id], :as row}]
  (assoc row :dashboard (select/select-one :models/Dashboard :toucan/pk dashboard-id)))

Hydrating Multiple Keys

You can hydrate several keys at one time:

(hydrate {...} :a :b)
  -> {:a 1, :b 2}

Nested Hydration

You can do recursive hydration by listing keys inside a vector:

(hydrate {...} [:a :b])
  -> {:a {:b 1}}

The first key in a vector will be hydrated normally, and any subsequent keys will be hydrated inside the corresponding values for that key.

(hydrate {...}
         [:a [:b :c] :e])
  -> {:a {:b {:c 1} :e 2}}

Flowchart

If you're digging in to the details, this is a flowchart of how hydration works:

                      hydrate <-------------+
                        |                   |
                    hydrate-forms           |
                        | (for each form)   |
                    hydrate-one-form        | (recursively)
                        |                   |
             keyword? --+-- sequence?       |
                |             |             |
          hydrate-key   hydrate-key-seq ----+
                |
       (for each strategy) <--------+
        ::automagic-batched         |
        ::multimethod-batched       |
        ::multimethod-simple        | (try next strategy)
                |                   |
       can-hydrate-with-strategy?   |
                |                   |
         yes ---+--- no ------------+
          |
 hydrate-with-strategy

batched-hydrate clj

(batched-hydrate model k instances)

Hydrate the key k in one or more instances of model. Implement this method to support batched hydration for the key k. Example implementation:

;; This method defines a batched hydration strategy for the :bird-type key for all models.
(m/defmethod hydrate/batched-hydrate [:default :is-bird?]
  [_model _k rows]
  ;; fetch a set of all the non-nil bird IDs in rows.
  (let [bird-ids           (into #{} (comp (map :bird-id) (filter some?)) rows)
        ;; if bird-ids is non-empty, fetch a map of bird ID => bird type
        bird-id->bird-type (when (seq bird-ids)
                             (select/select-pk->fn :bird-type :models/bird :id [:in bird-ids]))]
    ;; for each row add a :bird-type key.
    (for [row rows]
      (assoc row :bird-type (get bird-id->bird-type (:bird-id row))))))

Batched hydration implementations should try to be efficient, e.g. minimizing the number of database calls made rather than doing one call per row. If you just want to hydrate each row independently, implement simple-hydrate instead. If you are hydrating entire instances of some other model, consider setting up automagic batched hydration using model-for-automagic-hydration and possibly fk-keys-for-automagic-hydration.

can-hydrate-with-strategy? clj

(can-hydrate-with-strategy? model strategy k)

Can we hydrate the key k in instances of model using a specific hydration strategy?

Normally you should never need to call this yourself. The only reason you would implement it is if you are implementing a custom hydration strategy.

fk-keys-for-automagic-hydration clj

(fk-keys-for-automagic-hydration original-model dest-key hydrated-model)

The keys in we should use when automagically hydrating the key dest-key in instances of original-model with instances hydrated-model.

;; when hydrating :user in an order with a user, fetch the user based on the value of the :user-id key
;;
;; This means we take the value of :user-id from our order and then fetch the user with the matching primary key
;; (by default, :id)
(fk-keys-for-automagic-hydration :models/orders :user :models/user) => [:user-id]

The model that we are hydrating with (e.g. :models/user) is determined by model-for-automagic-hydration.

By default fk-keys-for-automagic-hydration is just the key we're hydrating, with -id appended to it; e.g. if we're hydrating :user the default FK key to use is :user-id. If this convention is fine, you do not need to implement this method. If you want to do something that does not follow this convention, like hydrate a :user key based on values of :creator-id, you should implement this method.

Example implementation:

;; hydrate orders :user key using values of :creator-id
(m/defmethod hydrate/fk-keys-for-automagic-hydration [:model/orders :user :default]
  [_original-model _dest-key _hydrated-model]
  [:creator-id])

Tips

When implementing this method, you probably do not want to specialize on the hydrated-model (the third part of the dispatch value) -- the model to use is normally determined by model-for-automagic-hydration. The only time you might want to specialize on hydrated-model is you wanted to do something like

;; by default when we hydrate the :user key with a :models/user, use :creator-id as the source FK
(m/defmethod hydrate/fk-keys-for-automagic-hydration [:default :model/orders :user :models/user]
  [_original-model _dest-key _hydrated-model]
  [:creator-id])

hydrate clj

(hydrate instance-or-instances)

(hydrate instance-or-instances & ks)

Hydrate the keys ks in a single instance or sequence of instances. See toucan2.tools.hydrate for more information.

hydrate-with-strategy clj

(hydrate-with-strategy model strategy k instances)

Hydrate the key k in instances of model using a specific hydration strategy.

Normally you should not call this yourself. The only reason you would implement this method is if you are implementing a custom hydration strategy.

model-for-automagic-hydration clj

(model-for-automagic-hydration original-model k)

The model that should be used to automagically hydrate the key k in instances of original-model.

(model-for-automagic-hydration :some-table :user) :-> :myapp.models/user

Dispatches off of the [[toucan2.protocols/model]] of the instance being hydrated and the key k that we are attempting to hydrate. To hydrate the key k for any model, you can use :default in your defmethod dispatch value. Example implementation:

 ;; when hydrating the :user key for *any* model, hydrate with instances of the model :models/user
 ;;
 ;; By default, this will look for values of :user-id in the instances being hydrated and then fetch the instances of
 ;; :models/user with a matching :id
 (m/defmethod hydrate/model-for-automagic-hydration [:default :user]
   [_original-model _k]
   :models/user)

 ;; when hydrating the :user key for instances of :models/orders, hydrate with instances of :models/user
 (m/defmethod hydrate/model-for-automagic-hydration [:models/orders :user]
   [_original-model _k]
   :models/user)

Automagic hydration looks for the fk-keys-for-automagic-hydration in the instance being hydrated, and if they're all non-nil, fetches instances of model-for-automagic-hydration with the corresponding toucan2.model/primary-keys. Thus you might also want to implement

• fk-keys-for-automagic-hydration for the model whose instances hydrating and the key being hydrated (e.g. :models/orders and :user)

• toucan2.model/primary-keys for the model you want to hydrate with (e.g. :models/user)

Tips

You probably don't want to write an implementation for [<some-model> :default] or [:default :default], unless you want every key that we attempt to hydrate to be hydrated by that method.

simple-hydrate clj

(simple-hydrate model k row)

Implementations should return a version of map row with the key k added.