Sequential core-set Monte Carlo (Beronov, Weilbach, Wood & Campbell, UAI 2021): SMC over data arriving in batches whose rejuvenation moves cost a bounded amount however much data has arrived.
The target after t batches is p_t(θ) ∝ p₀(θ) · Π_{s≤t} Π_b p(x_b⁽ˢ⁾ | θ).
Plain resample-move SMC (IBIS; Chopin 2002) must score every past datum in
each move, so step t costs O(t). SCMC keeps at most :memory M past data
points with weights — a core-set — whose weighted log-likelihood
approximates that of all data so far, and moves the particles against
log p₀(θ) + Σ_j w_j log p(u_j | θ).
After each step's moves, when the core-set and the new batch exceed M, it is recompressed: a sparse nonnegative least-squares fit (GIGA; Campbell & Broderick 2018) of the core-set's log-likelihood functions, centred and evaluated at the current particles, to their weighted sum. The fit is best where the population has its mass.
The reweighting by each new batch is exact. Only the moves target the
core-set approximation, so the particles carry its error: small when the
log-likelihoods compress well (exchangeable data, few parameters), larger
for a small :memory. With M at least the amount of data, SCMC is
resample-move SMC.
The engine is foerster.population/theta-smc on θ vectors: a step
resamples, then moves every particle by population-scaled random-walk
Metropolis.
Sequential core-set Monte Carlo (Beronov, Weilbach, Wood & Campbell, UAI
2021): SMC over data arriving in batches whose rejuvenation moves cost a
bounded amount however much data has arrived.
The target after t batches is p_t(θ) ∝ p₀(θ) · Π_{s≤t} Π_b p(x_b⁽ˢ⁾ | θ).
Plain resample-move SMC (IBIS; Chopin 2002) must score every past datum in
each move, so step t costs O(t). SCMC keeps at most `:memory` M past data
points with weights — a core-set — whose weighted log-likelihood
approximates that of all data so far, and moves the particles against
log p₀(θ) + Σ_j w_j log p(u_j | θ).
After each step's moves, when the core-set and the new batch exceed M, it
is recompressed: a sparse nonnegative least-squares fit (GIGA; Campbell &
Broderick 2018) of the core-set's log-likelihood functions, centred and
evaluated at the current particles, to their weighted sum. The fit is best
where the population has its mass.
The reweighting by each new batch is exact. Only the moves target the
core-set approximation, so the particles carry its error: small when the
log-likelihoods compress well (exchangeable data, few parameters), larger
for a small `:memory`. With M at least the amount of data, SCMC is
resample-move SMC.
The engine is `foerster.population/theta-smc` on θ vectors: a step
resamples, then moves every particle by population-scaled random-walk
Metropolis.(giga ls target m)Greedy iterative geodesic ascent (Campbell & Broderick 2018, Alg. 1):
nonnegative weights w over the vectors ls (double arrays of one length),
at most m of them nonzero, with Σ w_n·ls_n close to target.
It works on the unit sphere: each iteration adds the vector whose geodesic from the current direction best aligns with the geodesic to the target's direction, with the step that maximizes the alignment; the weights are scaled optimally at the end. A vector of norm 0 gets weight 0. Returns the weights as a vector.
Greedy iterative geodesic ascent (Campbell & Broderick 2018, Alg. 1): nonnegative weights w over the vectors `ls` (double arrays of one length), at most `m` of them nonzero, with Σ w_n·ls_n close to `target`. It works on the unit sphere: each iteration adds the vector whose geodesic from the current direction best aligns with the geodesic to the target's direction, with the step that maximizes the alignment; the weights are scaled optimally at the end. A vector of norm 0 gets weight 0. Returns the weights as a vector.
(scmc {:keys [prior-sample prior-log-density log-lik batches]}
&
[{:keys [particles memory scale seed]
:or {particles 1000 memory 100 scale 2.38}}])SCMC of the model given by
:prior-sample (fn []) θ ~ p₀, a vector
:prior-log-density (fn [θ]) log p₀(θ)
:log-lik (fn [θ datum]) log p(datum | θ)
:batches the data, a sequence of batches (sequences of data),
consumed lazily: a stream of batches runs in memory
bounded by the core-set and the population
with options :particles (1000), :memory M (100), :scale (2.38, the
random walk in population standard deviations), :seed.
Returns {:particles [θ …] (equally weighted) :log-z (the evidence estimate,
its increments exact, its moves approximate) :coreset [[w datum] …] (at
most M) :steps :max-coreset-size (the largest core-set the moves used,
before recompression: at most M plus a batch) :moves
:accepted}. Throws ::zero-evidence when a batch has zero likelihood
under every particle.
SCMC of the model given by
:prior-sample (fn []) θ ~ p₀, a vector
:prior-log-density (fn [θ]) log p₀(θ)
:log-lik (fn [θ datum]) log p(datum | θ)
:batches the data, a sequence of batches (sequences of data),
consumed lazily: a stream of batches runs in memory
bounded by the core-set and the population
with options `:particles` (1000), `:memory` M (100), `:scale` (2.38, the
random walk in population standard deviations), `:seed`.
Returns {:particles [θ …] (equally weighted) :log-z (the evidence estimate,
its increments exact, its moves approximate) :coreset [[w datum] …] (at
most M) :steps :max-coreset-size (the largest core-set the moves used,
before recompression: at most M plus a batch) :moves
:accepted}. Throws `::zero-evidence` when a batch has zero likelihood
under every particle.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |