Incremental Collections

Spindel tracks structural changes to collections as deltas, enabling O(delta) updates instead of O(n) re-computation. This is the foundation for efficient reactive UIs and incremental data processing.

Deltaable Collections

Deltaable collections wrap standard Clojure vectors, maps, and sets with top-level change tracking.

(require '[org.replikativ.spindel.incremental.deltaable :as d])

Creating Deltaable Collections

;; Vector
(def dv (d/deltaable-vector [1 2 3]))

;; Map
(def dm (d/deltaable-map {:a 1 :b 2}))

;; Set
(def ds (d/deltaable-set #{:x :y :z}))

Operations Produce Deltas

Standard Clojure operations on deltaable collections produce delta records:

(def dv (d/deltaable-vector [1 2 3]))

(def dv2 (conj dv 4))
(d/get-deltas dv2)
;; => [{:delta :add :path [3] :value 4}]

(def dv3 (assoc dv 0 10))
(d/get-deltas dv3)
;; => [{:delta :update :path [0] :value 10 :old-value 1}]

(def dv4 (pop dv))
(d/get-deltas dv4)
;; => [{:delta :remove :path [2] :value 3}]

Supported Operations by Type

Operation	Vector	Map	Set
conj	add at end	add key	add member
assoc	update at index	add/update key	—
dissoc	—	remove key	—
disj	—	—	remove member
pop	remove last	—	—
update	—	update key	—

Delta Format

{:delta    :add/:update/:remove  ;; operation type
 :path     [index-or-key]        ;; location
 :value    new-value             ;; the new value
 :old-value old-value}           ;; only for :update

Accessing Deltas

(d/get-deltas dv2)     ;; get accumulated deltas
(d/has-deltas? dv2)    ;; true if non-empty deltas
(d/clear-deltas dv2)   ;; return copy with deltas cleared
@dv2                   ;; get underlying raw value
(d/unwrap dv2)         ;; also gets raw value

Signal Auto-Wrapping

Signals automatically wrap collections as deltaable when you use standard operations:

(def items (signal []))

(binding [ec/*execution-context* ctx]
  (swap! items conj {:name "Alice"})   ;; auto-wrapped as deltaable
  (swap! items conj {:name "Bob"}))

;; Inside a spin, track gives you deltas:
(spin
  (let [{:keys [new old deltas]} (track items)]
    ;; deltas: [{:delta :add :path [1] :value {:name "Bob"}}]
    ))

Intervals

An Interval packages the old value, new value, and deltas from a signal change. It's what track returns.

(require '[org.replikativ.spindel.incremental.interval :as iv])

Structure

{:old    previous-value    ;; value at last spin execution
 :new    current-value     ;; value now
 :deltas [delta-records]}  ;; structural changes

Creating Intervals

(iv/interval 42)                 ;; static (no old, no deltas)
(iv/interval old-val new-val)    ;; with old and new
(iv/interval old new deltas)     ;; with explicit deltas

Querying Intervals

(iv/changed? interval)   ;; true if old != new
(iv/static? interval)    ;; true if no old and no deltas
(iv/interval? x)         ;; type check

Destructuring

Intervals support both map and sequential destructuring:

;; Map destructuring
(let [{:keys [new old deltas]} (track sig)] ...)

;; Sequential destructuring
(let [[new-val old-val deltas] (track sig)] ...)

;; Deref for just the current value
@(track sig)  ;; => current value

Merging Intervals

merge-intervals combines two intervals, preserving the oldest baseline and concatenating deltas:

(def iv1 (iv/interval :a :b [{:delta :add :path [0] :value :b}]))
(def iv2 (iv/interval :b :c [{:delta :update :path [0] :value :c :old-value :b}]))

(iv/merge-intervals iv1 iv2)
;; => {:old :a, :new :c, :deltas [compacted-deltas...]}

This is associative: merge(merge(a,b),c) = merge(a,merge(b,c)). Used by accumulate to preserve deltas under rate control.

Typed Delta Algebra

Combinators output a typed interval — a map carrying the value, its predecessor, the deltas, and the algebra under which those deltas compose and apply:

{:algebra <delta-algebra>
 :old     <previous value>
 :new     <current value>
 :deltas  <algebra-specific delta record, or nil, or the algebra's identity>}

The same accessors (iv/get-new, iv/get-old, iv/get-deltas, :new, :old, :deltas) work uniformly on both the Interval deftype returned by track and the typed-interval map returned by combinators.

The 3-state :deltas contract

:deltas carries one of three states with distinct meanings:

Value	Meaning	Consumer action
nil	"I don't know what changed."	Recompute from :new.
algebra identity (e.g. (a/empty-deltas alg))	"I verified — nothing changed."	Keep cached output.
populated Δ (subject to apply(:old, Δ) = :new)	"These changes happened."	Apply Δ incrementally.

Use iv/no-change? to test for the verified-no-change state across both legacy and typed intervals.

Algebras

Two algebras ship in org.replikativ.spindel.incremental.{sequence_algebra, map_algebra}:

• SequenceAlgebra — deltas are records of {:degree :grow :shrink :permutation :change :freeze}, applied in order grow → permutation → shrink → change → freeze. Output of imap / ifilter / islice / for-each* / iflat-map.
• MapAlgebra — deltas are {:assoc {k v} :dissoc #{k} :update {k {:algebra A :delta Δ}}}. Inner deltas under :update carry their own algebra, enabling nested incremental updates one level deep.

A scalar algebra (a/scalar-algebra) covers single-value outputs like ireduce's result.

Each algebra forms a monoid (D, ·, id) with apply / compose / empty operations; combinators rely on these laws (associativity, identity) so that delta pipelines compose without re-running source computations.

Incremental Combinators

Transform intervals incrementally — O(delta) work instead of O(n).

(require '[org.replikativ.spindel.incremental.combinators :as ic])

ifilter — Incremental Filter

Filter a collection incrementally. Items entering or leaving the filtered set generate appropriate deltas:

(spin
  (let [items (track items-signal)
        active (ic/ifilter :active? items)]
    ;; active is an Interval with filtered deltas
    ;; On each signal change, only processes changed items
    @active))

imap — Incremental Map

Transform items incrementally:

(spin
  (let [items (track items-signal)
        names (ic/imap :name items)]
    @names))  ;; => vector of names, updated incrementally

ireduce — Incremental Reduce

Maintain a running reduction over the source. When the source changes, the fold is recomputed from :new (cheap when rf is cheap; wrap an expensive per-item input in for-each* for per-element memoisation).

(spin
  (let [prices (track prices-signal)
        total (ic/ireduce + 0 prices)]
    @total))

ifor-each — Keyed Transformation

Transform items by key, only re-transforming changed items:

(spin
  (let [items (track items-signal)
        rendered (ic/ifor-each
                   :id                                    ;; key function
                   (fn [item] (render-item item))         ;; transform
                   items)]
    @rendered))

Reactive per-item spins

If render-fn returns spins rather than plain values, ifor-each's DOM-side equivalent (dom/foreach) returns a single outer spin that awaits every per-item spin and assembles a KeyedFragment. When the per-item spins are themselves reactive (they track signals and re- complete on signal change), the outer loop-spin's (await per-item-spin) re-fires for each re-completion and rebuilds the fragment.

The rebuild path re-reads the keyed cache fresh inside the loop-spin at fragment-build time rather than using the prev-by-key/prev-order that were captured at for-each* call time. The captured values are stale (typically empty on the first call); diffing against them would re-emit :add deltas for items that already exist and duplicate DOM nodes. Re-reading the cache fresh gives the true previous order and yields :update deltas for value changes, leaving the DOM stable.

This matters for any caller wiring ifor-each to a collection of reactive children — a sidebar of (ifor-each :kb/id kbs (fn [kb] (spin (track kb-state) …)))-style structures, for example.

islice — Windowed View

Maintain a window into a collection for virtual scrolling:

(spin
  (let [all-items (track items-signal)
        ;; window is a map {:start n :end n} — the islice macro
        ;; destructures it; passing a vector will fail to compile.
        visible (ic/islice {:start start-idx :end end-idx} all-items)]
    ;; Only processes items entering/leaving the window
    (iv/get-new visible)))

Delta Transducers

For streaming delta processing:

;; Transform delta values
(d/map-delta (fn [d] (update d :value str/upper-case)))

;; Filter deltas
(d/filter-delta (fn [d] (= :add (:delta d))))

;; Remove deltas (inverse of filter)
(d/remove-delta (fn [d] (= :remove (:delta d))))

;; Keep (transform + filter)
(d/keep-delta (fn [d] (when (= :add (:delta d)) (update d :value inc))))

;; Apply deltas to rebuild a collection
(reduce d/apply-delta [] deltas)

;; Compact redundant operations
(d/compact-deltas deltas)
;; Optimizations:
;;   - Multiple updates to same path → keep last
;;   - Add then remove → cancel out
;;   - Remove then add → convert to update

Transduce

(d/transduce-deltas
  (comp (d/filter-delta #(= :add (:delta %)))
        (d/map-delta #(update % :value str/upper-case)))
  []
  deltas)

Pattern: Track and Pipe into Typed Combinators

The typical incremental processing pattern is not to hand-walk raw deltas. The typed delta algebra exists so consumers can declare what they want and let combinators carry the deltas through:

(def items (signal (d/deltaable-vector [])))

(spin
  (let [items-iv (track items)
        ;; Filter + map without ever touching the delta vocabulary;
        ;; the typed combinators emit SequenceAlgebra `:seq-diff`
        ;; records, which the DOM discharge layer consumes directly.
        visible  (ic/ifilter active? items-iv)
        rendered (ic/imap render-row visible)]
    (el/div {:class "list"}
      (foreach/ifor-each :id rendered identity))))

If you really need a custom non-DOM consumer of a deltaable-collection signal's raw deltas (e.g. mirroring to an external system), you can walk them — but treat that as the low-level escape hatch:

;; Escape-hatch: raw delta walk. Only valid on deltaable-collection
;; output (the input edge of the pipeline). Typed-combinator outputs
;; carry algebra records, not this vocabulary.
(spin
  (let [iv (track items)
        new (iv/get-new iv)
        deltas (iv/get-deltas iv)]
    (if (empty? deltas)
      (full-render new)
      (doseq [{:keys [delta path value old-value]} deltas]
        (case delta
          :add    (insert-at-path path value)
          :remove (remove-at-path path)
          :update (update-at-path path value old-value))))))

See Also

• Effects — track returns intervals
• Combinators — accumulate with merge-intervals
• Getting Started — Basic signal usage