reflet — io.zalky/reflet 0.3.0-rc1

Liking cljdoc? Tell your friends :D

reflet.db

cljs

Provides an event sourced, reactive db, with mutation and query methods.

Storing data in graph form has many benefits for application and data design. The main trade-off is the performance cost of normalizing data during writes, and de-normalizing data during queries. Specifically, some graph queries can end up spanning the entire db, and so in a reactive application, understanding when to re-run queries is key to performance.

The naive approach of just re-running every query whenever the db changes is rarely feasible. Just one expensive query will grind the entire app to a halt, and predicting the cost of a query is a run-time calculation that depends on the data.

A key insight that can help optimize query denormalization is that g for the set of queries that conform to EQL, a result cannot change if the entities that were traversed while walking the joins have not changed.

An approach used elsewhere that leverages this guarantee is to use reagent reactions to cache entity joins in the computed result. This avoids recomputing the entire query if some of the entities that were traversed in the joins have not changed. Only the subgraph that depends on the changed entities will be recomputed.

Unfortunately this approach runs into a fundamental problem: by turning every join into a reactive computation, the cumulative overhead of tracking every reaction significantly exceeds the cost of just running the full non-reactive query. Equality checks are expensive, and the number of join reactions created depends on the shape of data at run-time, which can be huge. The naive approach, to just recompute the full query when the db changes, is always faster.

The solution presented in this namespace leverages the guarantee on query results and combines it with an event sourcing strategy to implement a differential, reactive loop between the db and queries.

To summarize: each query tracks the entities it traversed to produce its result. When a mutation operation is performed on the db, the entities that were touched by the mutation are then directly mapped to the queries that need to update. This is essentially a mutation event sourcing strategy.

Importantly, while queries react to mutation events, they will also always return the correct value if for whatever reason the db is reset outside the mutation functions, for example, if the db value is set by Re-frame-undo or debug tooling like Re-frame-10x. The reactive queries respect time-travel.

Another key design point is that query operations must be commutative and idempotent with respect to the tracking index. On the mutation side, any set of db operations that were previously commutative or idempotent on non-graph data should remain so.

The db mutation operations are pure functions that operate on the db value provided to event handlers. They can be arbitrarily composed in event handler code, similar to associative methods like clojure.core/assoc. They can also be freely interleaved with non-normalzied db operations. At the end of the day the Re-frame db is still just a big map.

It is important to note that the query index is not actually stored as part of application state, since it is fundamentally not application state. Rather, it is state associated with the reactive queries, analogous to how reagent reactions maintain internal state to function properly. If you wind back application state to a previous value, the query state should not change until the queries have re-run.

In order for the pure db mutation functions to operate on the index, an interceptor injects the index into the db coeffect, as well as removes and commits the updated index as part of the :after interceptor chain.

Three iteration ticks drive the reactive algorithm:

db tick
index tick
query tick one for each query

Step by step:

The db tick and index tick are incremented synchronously by the database mutation functions. If at any point the db tick and index tick do not match, this indicates time travel, and the query index is flushed by the mutation function.
If an entity is touched by a db mutation function, then any query that was tracking that entity is marked as touched in the query index.
During the subscription phase, a query must be re-run iff one or more of the following is true:

a) the query is marked as touched in the query index, b) the query's tick is nil, indicating a fresh query index, or c) the db tick is not equal to the index tick, indicating time travel
This conditional constitutes the ::query-tick signal. The signal returns the current query tick if the query does not need to re-run, or the current db tick if it does.
When a query is re-run, it: a) updates the entities it is tracking in the query index b) clears the touched query set c) syncs the query tick to the db tick
After all queries have been run, the index tick is synced to to the db tick.

The above algorithm will only react to db changes via the db mutation functions. Specifically, the graph data stored at the ::data key of the Re-frame db should never be directly manipulated, except via the mutation functions.

However, aside from this one constraint, all regular db operation on non-graph data are completely orthogonal to the differential reactive loop. At the end of the day, the db is still just a map, and you can mutate it in any way as long as you persist the ::data key.

This namespace additionally provides an implementation for link queries similar to those in Fulcro or Om Next. A link attribute allows you to store graph data at a semantically meaningful, global keyword in the graph db. For example, you could store graph user data at a global ::current-user attribute in the db index.

Finally, this algorithm is implemented entirely via existing reagent and Re-frame machinery.

Provides an event sourced, reactive db, with mutation and query
methods.

Storing data in graph form has many benefits for application and
data design. The main trade-off is the performance cost of
normalizing data during writes, and de-normalizing data during
queries. Specifically, some graph queries can end up spanning the
entire db, and so in a reactive application, understanding when to
re-run queries is key to performance.

The naive approach of just re-running every query whenever the db
changes is rarely feasible. Just one expensive query will grind the
entire app to a halt, and predicting the cost of a query is a
run-time calculation that depends on the data.

A key insight that can help optimize query denormalization is that g
for the set of queries that conform to EQL, a result cannot change
if the entities that were traversed while walking the joins have not
changed.

An approach used elsewhere that leverages this guarantee is to use
reagent reactions to cache entity joins in the computed result. This
avoids recomputing the entire query if some of the entities that
were traversed in the joins have not changed. Only the subgraph that
depends on the changed entities will be recomputed.

Unfortunately this approach runs into a fundamental problem: by
turning every join into a reactive computation, the cumulative
overhead of tracking every reaction significantly exceeds the cost
of just running the full non-reactive query. Equality checks are
expensive, and the number of join reactions created depends on the
shape of data at run-time, which can be huge. The naive approach, to
just recompute the full query when the db changes, is always faster.

The solution presented in this namespace leverages the guarantee on
query results and combines it with an event sourcing strategy to
implement a differential, reactive loop between the db and queries.

To summarize: each query tracks the entities it traversed to produce
its result. When a mutation operation is performed on the db, the
entities that were touched by the mutation are then directly mapped
to the queries that need to update. This is essentially a mutation
event sourcing strategy.

Importantly, while queries react to mutation events, they will also
always return the correct value if for whatever reason the db is
reset outside the mutation functions, for example, if the db value
is set by Re-frame-undo or debug tooling like Re-frame-10x. The
reactive queries respect time-travel.

Another key design point is that query operations must be
commutative and idempotent with respect to the tracking index. On
the mutation side, any set of db operations that were previously
commutative or idempotent on non-graph data should remain so.

The db mutation operations are pure functions that operate on the db
value provided to event handlers. They can be arbitrarily composed
in event handler code, similar to associative methods like
`clojure.core/assoc`. They can also be freely interleaved with
non-normalzied db operations. At the end of the day the Re-frame db
is still just a big map.

It is important to note that the query index is not actually stored
as part of application state, since it is fundamentally not
application state. Rather, it is state associated with the reactive
queries, analogous to how reagent reactions maintain internal state
to function properly. If you wind back application state to a
previous value, the query state should not change until the queries
have re-run.

In order for the pure db mutation functions to operate on the index,
an interceptor injects the index into the db coeffect, as well as
removes and commits the updated index as part of the :after
interceptor chain.

Three iteration ticks drive the reactive algorithm:

1. db tick
2. index tick
3. query tick one for each query

Step by step:

1. The db tick and index tick are incremented synchronously by
   the database mutation functions. If at any point the db tick
   and index tick do not match, this indicates time travel, and
   the query index is flushed by the mutation function.

2. If an entity is touched by a db mutation function, then any
   query that was tracking that entity is marked as touched in
   the query index.

3. During the subscription phase, a query must be re-run iff one or
   more of the following is true:

   a) the query is marked as touched in the query index,
   b) the query's tick is nil, indicating a fresh query index, or
   c) the db tick is not equal to the index tick, indicating
      time travel

4. This conditional constitutes the ::query-tick signal. The signal
   returns the current query tick if the query does not need to
   re-run, or the current db tick if it does.

5. When a query is re-run, it:
   a) updates the entities it is tracking in the query index
   b) clears the touched query set
   c) syncs the query tick to the db tick

6. After all queries have been run, the index tick is synced to
   to the db tick.

The above algorithm will only react to db changes via the db
mutation functions. Specifically, the graph data stored at
the ::data key of the Re-frame db should never be directly
manipulated, except via the mutation functions.

However, aside from this one constraint, all regular db operation on
non-graph data are completely orthogonal to the differential
reactive loop. At the end of the day, the db is still just a map,
and you can mutate it in any way as long as you persist the ::data
key.

This namespace additionally provides an implementation for link
queries similar to those in Fulcro or Om Next. A link attribute
allows you to store graph data at a semantically meaningful, global
keyword in the graph db. For example, you could store graph user
data at a global `::current-user` attribute in the db index.

Finally, this algorithm is implemented entirely via existing reagent
and Re-frame machinery.

raw docstring

reflet.db.normalize

Provides data normalization functions.

Provides data normalization functions.

raw docstring

reflet.db.tx

reflet.debug

disp-tap

reflet.debug.glyphs

reflet.debug.preload

No vars found in this namespace.

reflet.debug.ui

reflet.debug.ui.data

reflet.debug.ui.data.impl

reflet.fsm

Provides finite state machine DSL and implementation.

This FSM implementation is based on an entity model, where any entity in the db can be transitioned through allowed states given defined inputs. This means both domain entities as well as component entities can be used as FSMs.

Each FSM is defined declaratively. For example:

(f/reg-fsm ::review (fn [self user] {:ref self :stop #{::accepted ::cancelled} :fsm {nil {[:voted self] ::review} ::review {:* {:to ::decision :pull [self] :when ::review-threshold :dispatch [:notify user]}} ::decision {[::fsm/timeout self 1000] ::cancelled [:accepted self] {:to ::accepted} [:revisit self] ::review}}}))

This spec requires the following attributes:

:ref A db reference to the FSM entity being advanced through states.

:fsm Defines the allowed states of the FSM, mapping each state to a set of allowed transitions.

The following optional attributes are also supported:

:attr The entity attribute where the FSM state is stored. If not provided ::state is used.

:stop One or more states which when reached will stop the FSM. The FSM interceptor will be removed, but the FSM state will also be set to the relevant state.

:return The result returned by the FSM subscription. This is expressed as a pull spec that is run against the db with the FSM :ref as the root entity. The default pull spec returns a single attribute, which is the FSM state attribute (see the :attr option above).

:or Advance the FSM to some default state if there is no state value in the db on init.

:dispatch One or more events to dispatch immediately after starting the FSM.

:dispatch-later Same, but conforms to the Re-frame :dispatch-later fx syntax.

Like Re-frame events and subscriptions, FSMs are uniquely identified by a vector that starts with their reg-fsm id, followed by their arguments: [::review self].

FSMs are implemented via interceptors that will advance the FSM on any received Re-frame event.

A running FSM will throw an error if it ever reaches a state that is not defined in either the :fsm state map, or the set of :stop states. You can define a state with no transitions by mapping the state to nil:

{:fsm {::going-nowhere nil ::going-somewhere {[:event ref] ::going-nowhere}}}

Note this does NOT mean that ::going-nowhere will transition to the nil state, but rather that ::going-nowhere has no transitions defined.

This will effectively pause the FSM, without actually turning off its interceptor. To prevent the interceptor from running needlessly you would normally declare ::going-nowhere as a stop state instead:

{:stop #{::going-nowhere} :fsm {::going-somewhere {[:event ref] ::going-nowhere}}}

The one use case for mapping a state to nil, rather than declaring it as a stop state, is if you want the FSM to sit and do nothing until some other process has manually set the state attribute of the FSM entity in the db. At this point the FSM would immediately advance through the new state's defined transitions, starting with the same event that manually set the FSM state. But this use case is pretty niche.

Each transition for a given state is a map between an input event, and one or more output clauses.

There are two different types of event inputs currently defined:

Event transitions
Timeout transitions

Event transitions match a received event against a set of event prefixes. Each prefix either matches the event exactly, or an event with additional args. If more than one event prefix would match, then the longest matching prefix is chosen. For example, given the received event:

[:voted self first-pref second-pref]

and the set of input prefixes:

[:voted self] [:voted self first-pref]

Then matching prefix would be:

[:voted self first-pref]

You can match any event with the special keyword :*.

Timeout transitions are just like event transitions, except the first three positional elements of their event vector are:

[:reflet.fsm/timeout ref ms ...]

where ref is an entity reference, and ms is the timeout duration in milliseconds.

If the FSM arrives at a state that has a timeout in its transition map, and the timeout's ref is the same as the FSM :ref and specifies a ms duration, then the FSM will immediately set a timeout for that state that will fire within the designated time. If no other event causes the FSM to advance in this time, the timeout event is fired via dispatch-sync to advance the FSM through the timeout transition. Any timeouts that do not fire are cleaned up appropriately. If the timeout's ref is not the same as the FSM :ref, or if the timeout does not define a ms duration, then the timeout will be matched just like a regular event. This way you can listen for timeouts from other FSMs.

Only one wildcard or timeout transition is allowed in a state's transition map. Otherwise, a state's transition map can have an arbitrary number of event transitions. If a state defines both a wildcard and a set of event transitions, the event transitions will always be matched before the wildcard transition. As before: the most specific match will win.

All transitions define one or more output clauses. Each output clause be expressed in either simple or expanded form.

Simple: Just a state keyword
Complex: A map containing the following attributes:

:to The next state to transition to [required]

:when A conditional clause that must return true for the transition to match. This can be either, a) the id of a Clojure spec that must return s/valid? true, or b) a Clojure function that returns a truthy value. By default, the input to either the spec or the function will be the received event that triggered the transition. If :pull is specified, a list of entities will be provided as input instead of the event. See the :pull option for more details. [optional]

:pull A list of entity references. These entities are then passed as inputs to the :when conditional, in place of the trigger event. This allows you to build FSMs that use the state of other FSMs as inputs to their transitions. [optional]

:dispatch An event vector to dispatch on a successful transition [optional]

:dispatch-later Same semantics as :dispatch-later Re-frame fx.

Once an FSM has been declared using reg-fsm, instances can be created via subscriptions. When created this way, an FSM's lifecycle is tied to the lifecycle of its subscription. When the subscription is disposed, the FSM is also stopped. Subscriptions are the preferred way to create FSMs.

If you really need to start or stop and FSM during the event/fx phase of the application, you can do so using the ::start and ::stop event and fx handlers, with the caveat that an FSM can never be started or stopped while mutating the db at the same time. The FSM implementation actually has an interceptor that throws an error if this ever happens.

When an FSM is started, its initial state will be whatever is referenced by its FSM :ref in the db. If no state exists, then it will be nil. You should always define a nil transition, or set the :to option to advance the FSM on startup. Otherwise, the FSM will never get out of the nil state.

Because the FSM implementation is based on global interceptors that run every time, all the matching and lookup algorithms are written to be very fast.

Provides finite state machine DSL and implementation.

This FSM implementation is based on an entity model, where any
entity in the db can be transitioned through allowed states given
defined inputs. This means both domain entities as well as component
entities can be used as FSMs.

Each FSM is defined declaratively. For example:

(f/reg-fsm ::review
  (fn [self user]
    {:ref  self
     :stop #{::accepted ::cancelled}
     :fsm  {nil        {[:voted self] ::review}
            ::review   {:* {:to       ::decision
                            :pull     [self]
                            :when     ::review-threshold
                            :dispatch [:notify user]}}
            ::decision {[::fsm/timeout self 1000] ::cancelled
                        [:accepted self]          {:to ::accepted}
                        [:revisit self]           ::review}}}))

This spec requires the following attributes:

`:ref`
          A db reference to the FSM entity being advanced through
          states.

`:fsm`
          Defines the allowed states of the FSM, mapping each state
          to a set of allowed transitions.

The following optional attributes are also supported:

`:attr`
          The entity attribute where the FSM state is stored. If not
          provided `::state` is used.

`:stop`
          One or more states which when reached will stop the
          FSM. The FSM interceptor will be removed, but the FSM
          state will also be set to the relevant state.

`:return`
          The result returned by the FSM subscription. This is
          expressed as a pull spec that is run against the db with
          the FSM `:ref` as the root entity. The default pull spec
          returns a single attribute, which is the FSM state
          attribute (see the `:attr` option above).

`:or`
          Advance the FSM to some default state if there is no
          state value in the db on init.

`:dispatch`
          One or more events to dispatch immediately after starting
          the FSM.

`:dispatch-later`
          Same, but conforms to the Re-frame `:dispatch-later` fx
          syntax.

Like Re-frame events and subscriptions, FSMs are uniquely identified
by a vector that starts with their `reg-fsm` id, followed by their
arguments: `[::review self]`.

FSMs are implemented via interceptors that will advance the FSM on
any received Re-frame event.

A running FSM will throw an error if it ever reaches a state that is
not defined in either the `:fsm` state map, or the set of `:stop`
states. You can define a state with no transitions by mapping the
state to `nil`:

{:fsm {::going-nowhere   nil
       ::going-somewhere {[:event ref] ::going-nowhere}}}

Note this does NOT mean that `::going-nowhere` will transition to
the `nil` state, but rather that `::going-nowhere` has no
transitions defined.

This will effectively pause the FSM, without actually turning off
its interceptor. To prevent the interceptor from running needlessly
you would normally declare `::going-nowhere` as a stop state
instead:

{:stop #{::going-nowhere}
 :fsm  {::going-somewhere {[:event ref] ::going-nowhere}}}

The one use case for mapping a state to `nil`, rather than declaring
it as a stop state, is if you want the FSM to sit and do nothing
until some other process has manually set the state attribute of the
FSM entity in the db. At this point the FSM would immediately
advance through the new state's defined transitions, starting with
the same event that manually set the FSM state. But this use case is
pretty niche.

Each transition for a given state is a map between an input event,
and one or more output clauses.

There are two different types of event inputs currently defined:

1. Event transitions
2. Timeout transitions

Event transitions match a received event against a set of event
prefixes. Each prefix either matches the event exactly, or an event
with additional args. If more than one event prefix would match,
then the longest matching prefix is chosen. For example, given the
received event:

[:voted self first-pref second-pref]

and the set of input prefixes:

[:voted self]
[:voted self first-pref]

Then matching prefix would be:

[:voted self first-pref]

You can match any event with the special keyword :*.

Timeout transitions are just like event transitions, except the first
three positional elements of their event vector are:

[:reflet.fsm/timeout ref ms ...]

where `ref` is an entity reference, and `ms` is the timeout duration
in milliseconds.

If the FSM arrives at a state that has a timeout in its transition
map, and the timeout's `ref` is the same as the FSM `:ref` and
specifies a `ms` duration, then the FSM will immediately set a
timeout for that state that will fire within the designated time. If
no other event causes the FSM to advance in this time, the timeout
event is fired via `dispatch-sync` to advance the FSM through the
timeout transition. Any timeouts that do not fire are cleaned up
appropriately. If the timeout's `ref` is not the same as the FSM
`:ref`, or if the timeout does not define a `ms` duration, then the
timeout will be matched just like a regular event. This way you can
listen for timeouts from other FSMs.

Only one wildcard or timeout transition is allowed in a state's
transition map. Otherwise, a state's transition map can have an
arbitrary number of event transitions. If a state defines both a
wildcard and a set of event transitions, the event transitions will
always be matched before the wildcard transition. As before: the
most specific match will win.

All transitions define one or more output clauses. Each output
clause be expressed in either simple or expanded form.

1. Simple: Just a state keyword
2. Complex: A map containing the following attributes:

`:to`
          The next state to transition to [required]

`:when`
          A conditional clause that must return true for the
          transition to match. This can be either, a) the id of a
          Clojure spec that must return s/valid? `true`, or b) a
          Clojure function that returns a truthy value. By default,
          the input to either the spec or the function will be the
          received event that triggered the transition. If `:pull`
          is specified, a list of entities will be provided as input
          instead of the event. See the `:pull` option for more
          details. [optional]

`:pull`
          A list of entity references. These entities are then
          passed as inputs to the :when conditional, in place of the
          trigger event. This allows you to build FSMs that use the
          state of other FSMs as inputs to their transitions.
          [optional]

`:dispatch`
          An event vector to dispatch on a successful transition
          [optional]

`:dispatch-later`
          Same semantics as `:dispatch-later` Re-frame fx.

Once an FSM has been declared using `reg-fsm`, instances can be
created via subscriptions. When created this way, an FSM's lifecycle
is tied to the lifecycle of its subscription. When the subscription
is disposed, the FSM is also stopped. Subscriptions are the
preferred way to create FSMs.

If you really need to start or stop and FSM during the event/fx
phase of the application, you can do so using the `::start` and
`::stop` event and fx handlers, with the caveat that an FSM can
never be started or stopped while mutating the db at the same
time. The FSM implementation actually has an interceptor that throws
an error if this ever happens.

When an FSM is started, its initial state will be whatever is
referenced by its FSM `:ref` in the db. If no state exists, then it
will be `nil`. You should always define a `nil` transition, or set
the `:to` option to advance the FSM on startup. Otherwise, the FSM
will never get out of the `nil` state.

Because the FSM implementation is based on global interceptors that
run every time, all the matching and lookup algorithms are written
to be very fast.

raw docstring

reflet.interceptors

reflet.interop

Provides a DB for JS objects and DOM nodes.

Items are indexed by entity references obtained through the reflet.core/with-ref macro, and cleaned up when their React component unmounts. Do not use refs that were not created by with-ref unless you are prepared to clean them up yourself.

Provides a DB for JS objects and DOM nodes.

Items are indexed by entity references obtained through the
`reflet.core/with-ref` macro, and cleaned up when their React
component unmounts. Do not use refs that were not created by
`with-ref` unless you are prepared to clean them up yourself.

raw docstring

reflet.log

No vars found in this namespace.

reflet.poly

Provides polymorphic dispatch resolution.

Provides polymorphic dispatch resolution.

raw docstring

reflet.ref-spec

No vars found in this namespace.

reflet.trie

Trie for longest prefix matching.

Ideally this would implement clojure.lang interfaces.

Trie for longest prefix matching.

Ideally this would implement clojure.lang interfaces.

raw docstring

reflet.util.transients

cljdoc is a website building & hosting documentation for Clojure/Script libraries

Keyboard shortcuts Report a problem cljdoc on GitHub

× close

reflet.config

reflet.core

reflet.css.bundled

reflet.db

reflet.db.normalize

reflet.db.tx

reflet.debug

reflet.debug.glyphs

reflet.debug.preload

reflet.debug.ui

reflet.debug.ui.data

reflet.debug.ui.data.impl

reflet.debug.ui.impl

reflet.fsm

reflet.interceptors

reflet.interop

reflet.log

reflet.poly

reflet.ref-spec

reflet.trie

reflet.util.transients