A reference for every inference entry point in org.replikativ.foerster.core
(infer below). Each returns a spin resolving an EmpiricalMeasure (see
posteriors); at the REPL deref it inside
sp/with-context, inside a spin await it. For which to use when, see the
algorithms notebook.
infer/infer runs any method by name, as Anglican's doquery did:
(infer/infer (model) {:method :smc :particles 1000})
(infer/infer (model) {:method :mh :iterations 4000 :chains 4 :burn 1000})
(infer/infer (model) {:method :pmmh :particles 100 :iterations 2000 :params #{:sigma}})
Methods: :enumerate (exact, see below); :importance, :smc, :tempered (:particles); :pimh,
:pgibbs, :pgas, :ipmcmc, :bbvi (:particles, :iterations); :mh,
:rmh (:iterations per chain, :chains, :burn, :step-size); :kernel
(:kernel, :chains); :pmmh (:particles, :iterations, :params).
Other options go to the method's own function, described below.
Which to start with:
:smc with as many particles as you can afford; it also
estimates the evidence.:rmh or HMC on a
block, several chains, and check R-hat and ESS
(posteriors).:pmmh, or SMC² online;
a multimodal posterior: :tempered.{:method :enumerate} (org.replikativ.foerster.enumerate) computes the
exact posterior of a program whose latent choices all have finite support
(dist/support: Bernoulli, flip, discrete, categorical, binomial,
uniform-discrete): at every latent site the particle's world forks once per
value, each branch weighted by that value's probability. Branches share the
program up to where they part, so the cost is the number of complete
assignments (:max-branches, default 100 000, bounds it). Its measure has
one particle per assignment and its m/log-marginal is the exact evidence:
the reference for checking the other methods on small discrete models. A
latent site with infinite or continuous support is refused.
smc-infer, importance-sampling, pimh-infer, pgibbs-infer,
pgas-infer, ipmcmc-infer, bbvi-infer, and kernel-infer with a
non-Markov-chain kernel:
| Option | Default | Meaning |
|---|---|---|
:world-policy | :fresh | :fork runs the particles in canonical forks of the caller's world (worlds) |
:world-opts | {} | fork options for :fork: :systems (:all, :none or a set of system ids), :rights, :snapshots |
:authority, :grant | — | budgets under :fork (worlds) |
:executor | spindel's default | the executor the worlds run on |
:resample-threshold | 0.5 | SMC resamples when the effective sample size is below this fraction of the particles |
:policy | the prior | a foerster.trace/policy deciding the sites: constraints, interventions, custom proposals (extending) |
:anchors, :rejuvenate | — | resample-move (below); smc-infer and pimh-infer, fresh worlds |
:resampling | :systematic | :stratified, :residual or :multinomial (Douc, Cappé & Moulines 2005); smc-infer, pimh-infer |
:genealogy? | false | record every resampling's ancestor indices in the measure's :history |
:batch | — | B: resample in batches of B in arrival order instead of at a barrier for all (below) |
:smcp3 | — | {:forward K :backward L}: SMCP3 move-reweight steps at each observation (below); needs :anchors |
An option a method does not take is refused (::infer/unknown-options), and
so are :world-opts, :authority and :grant under :world-policy :fresh
(::infer/fork-only-options): an option that would be ignored is a mistake.
importance-sampling takes no :resample-threshold, BBVI no :policy or
:resample-threshold (it decides both), and the Markov-chain kernels only
:executor and :world-policy :fresh.
A program can also name a proposal at a site:
(sample (dist/normal 0 1) :id :mu :proposal guide) draws a fresh value
from guide — a distribution the program computes, for instance from its
data, which is how an amortized guide plugs in — and weighs it by
log p − log q, so every method's target stays the program's posterior.
Replays, moves and a policy's :draw take precedence.
| Function | What it does |
|---|---|
(smc-infer model n opts) | Sequential Monte Carlo with n particles, resampling at observations. The measure's m/log-marginal estimates the evidence. |
(importance-sampling model n opts) | n weighted runs, never resampled. |
(pimh-infer model n iterations opts) | Particle independent MH: iterations SMC sweeps of n particles, each accepted on its evidence estimate; every sweep's particles pooled. |
(pgibbs-infer model n iterations opts) | Particle Gibbs: iterated conditional SMC, each sweep keeping a trajectory drawn from the previous one; sweeps pooled. |
(pgas-infer model n iterations opts) | PGAS: particle Gibbs whose retained particle redraws its ancestor at every observation. Costs a replay of the retained future per particle per observation. |
(ipmcmc-infer model n iterations opts) | Interacting particle MCMC: :num-nodes (default 8) SMC nodes, :num-csmc-nodes (default half) of them conditional, exchanged by Gibbs updates on their evidence; :all-particles? (default true) emits every node's particles, false one per node. |
(bbvi-infer model n iterations opts) | Black-box variational inference: learns a mean-field q in iterations updates from n samples each. Each site's q starts as its prior when first reached, then moves by gradient only (it does not follow a prior that depends on other latents). :base-lr (1.0), :robbins-monro (0.0; step size base-lr / (t+1)^robbins-monro), :adagrad (true: AdaGrad; γ ∈ (0,1): RMSprop; false: plain steps). The measure holds n importance samples from the final q; (infer/get-variational-dists measure) returns q, {address distribution}. |
The pooled particle-MCMC measures are MCMC estimates: their weights are
normalized per sweep, and m/log-marginal of the pool is not an evidence
estimate.
(tempered-infer model n opts) (org.replikativ.foerster.tempering) moves a
population of complete program runs from the prior to the posterior through
the targets p(x)·L(x)^β, L the observations' and factors' likelihood, with β
rising from 0 to 1 (an SMC sampler; Del Moral, Doucet & Jasra 2006). Each
step chooses the next β so that the conditional ESS of the reweighting stays
at :ess-target·N (Zhou, Johansen & Aston 2016), resamples, and moves every
particle by Metropolis-Hastings at the new temperature — random walks scaled
by the population's spread on continuous sites, prior proposals elsewhere.
Use it for posteriors with separated modes, which a Markov chain does not cross and SMC in program order only reaches if its early particles happen to:
(infer/tempered-infer (model) 1000) ; β schedule in :temperatures
(infer/tempered-infer (model) 1000 {:waste-free 10}) ; Dau & Chopin 2022
| Option | Default | |
|---|---|---|
:ess-target | 0.5 | higher: more, smaller temperature steps |
:moves | one sweep | single-site MH moves per particle per step |
:scale | 2.38 | random-walk scale in population standard deviations |
:waste-free | — | P: resample N/P particles and keep every state of their P-step chains |
:max-steps | 1000 |
m/log-marginal estimates the evidence. Every move re-runs the program from
the moved site, so a step costs N sweeps of the model; fresh worlds only.
For data that arrive one observation at a time — IBIS (Chopin 2002) — use SMC with resample-move on the static parameters, below.
SMC resamples at each observation once every particle has arrived, so a
population whose steps take uneven time — a model call, a tool, a simulation
that sometimes runs long — waits for its slowest particle at every
observation. With :batch B, the particles arriving at an observation form
batches of B in arrival order; a full batch resamples within itself at once
(B children, each weighted by the batch's mean weight), and the last batch
of an observation closes once no other particle can still reach it.
(infer/smc-infer (model) 1000 {:batch 32})
Weights are never reset, so the evidence is the final mean weight. Each
batch's resampling keeps its total weight in expectation, whatever decided
which particles are in it, so the estimate stays unbiased even when the
arrival order depends on the particles' states (Paige et al. 2014's
condition for the particle cascade, here per batch; an island model with
islands formed by arrival). B = N is ordinary SMC; smaller B resamples
sooner but within fewer particles. The measure's :history lists every
batch. Batch membership follows arrival order, so on a multi-threaded
executor runs are not reproducible from their seed. Not with stream sites,
:retained, :anchors or :smcp3.
(cascade/cascade model n opts) (org.replikativ.foerster.cascade; Paige,
Wood, Doucet & Teh 2014) runs every particle on its own. At an observation a
particle compares its weight W with the running mean W̄ of the weights that
arrived there so far. Below the mean it survives with probability W/W̄, as
one copy weighted W̄; above it, it branches into ⌈W/W̄⌉ copies (forks of its
world) while the stage has produced no more children than min(n, arrivals),
⌊W/W̄⌋ otherwise, each weighted W/M — the paper's Eq. 14, whose feedback
keeps the population near n. Nobody waits for anybody; the evidence
estimate (1/n) Σ W over the particles that reach the end is unbiased
whatever order they arrive in.
| Option | Default | |
|---|---|---|
:cap | 4·n | most particles alive at once; copies beyond it fold into a multiplicity on the particle |
:concurrency | 32 | particles running between observations at once; the rest wait and resume in random order, which keeps the population stable (the paper's §4) |
:policy, :executor |
It ends when every particle has finished, never by wall clock, which would
favour the fast ones (Murray, Singh & Lee 2021). Each particle has one child
per observation on average, so the number that reach the end varies a lot
from run to run (a critical branching process); its :cascade reports
:launched, :finished, :peak and :collapsed. Runs on a
multi-threaded executor are not reproducible from their seed. Where the
population should stay fixed, :batch (above) is the steadier choice.
SMC resamples by copying good particles and dropping bad ones, so after a few observations most particles share their early choices (a static parameter sampled once is the extreme case). Resample-move (Gilks & Berzuini 2001) gives every particle Metropolis-Hastings moves after each resampling, which restores diversity without changing the target or the weights:
(infer/smc-infer (model) 1000
{:anchors #{:mu} ; where moves may start
:rejuvenate {:moves 2 ; per particle, per resampling
:propose (trace/random-walk-proposal 0.2)}})
A move replays the program from an anchor — a kept fork of the world at a latent site — up to the observation the particle is parked at, and accepts on the MH ratio of the two partial traces. Which sites keep anchors is the trade-off:
:anchors | Moves may target | Cost |
|---|---|---|
| a set of addresses | those sites (static parameters) | one anchor per particle per site; a move replays from the site to the current observation, O(t) |
{:lag L} | latent sites of the last L observations | anchors of older sites are released; a move replays O(L) steps — fixed-lag rejuvenation, and the choice for smc/stream |
:all | every latent site | an anchor per latent site, shared between particles of one ancestor |
(fn [sp]) | sites the predicate names |
Moves happen only when the population is resampled, so with the default
:resample-threshold they become rare as a static posterior concentrates
(Chopin 2002). The measure's :rejuvenation reports :moves, :accepted
and :max-anchors. Resample-move runs in fresh worlds only, and not in
conditional SMC (particle Gibbs, PGAS). A model whose moves reach effects
(a model call) runs them again on every replay.
SMCP3 (Lew et al. 2023) generalizes resample-move: at each observation a particle takes a step of a forward kernel K and is reweighted by a backward kernel L instead of accepting or rejecting. K may change any latent site the particle has passed — revise the past, propose from the data just seen — and the weight keeps the particle properly weighted:
w ← w · p(x')·L(u' | x') / (p(x)·K(u | x)) · |J|
(def random-walk-kernel
{:forward (fn [{mu :mu} trace] ; the particle's choices and trace
(let [u (dist/draw (dist/normal 0.0 0.05))]
{:updates {:mu (+ mu u)} ; new values
:log-q (dist/logpdf (dist/normal 0.0 0.05) u)
:reverse (- u)})) ; what L would draw to go back
:backward (fn [choices' u'] (dist/logpdf (dist/normal 0.0 0.05) u'))})
(infer/smc-infer (model) 1000 {:anchors #{:mu} :smcp3 random-walk-kernel})
K draws from foerster.random (it runs in the particle's stream), may
return :log-jacobian for a deterministic continuous map, and needs an
anchor at the earliest site it updates. Sites it does not update are kept,
and sites the replay reaches afresh are drawn from their prior. The weight's
variance depends on how well L reverses K: a symmetric random walk as L
multiplies the weights by π(x')/π(x), which is fine for steps small against
the posterior's spread and heavy-tailed for large ones; an L close to the
reversal of K under the target keeps them flat. Fresh worlds, not with
:retained.
A program whose proposals come from an outside process — a language model's
turn, a simulator — with no density to report, and whose data is a score
(a verifier, a judge, a process reward model), is steered by factors that
are barriers: (factor w :barrier true) makes SMC park and resample there,
as at an observation. foerster.steer/model builds the step loop:
(infer/smc-infer
(steer/model {:init s0
:step (fn [state] (spin …)) ; the next state, from the process
:value (fn [state] (spin …)) ; log ψ: the reward to come
:reward (fn [state] (spin …)) ; the final log potential
:done? (fn [state] …)})
16 {:resampling :stratified})
The target is p(trajectory)·exp(reward): the process's own law tilted by the
reward (Korbak et al. 2022). The process's randomness has no sample site, so
it acts as the proposal and its density cancels; the weights are the
factors. A value estimate twists the intermediate targets — each step adds
log ψ_t − log ψ_{t−1} and the end reward − log ψ_T — which resamples early
without changing the final target (twisted SMC); without one, SMC is
best-of-N weighted by the reward. The model starts at smc/start-site, so
every particle takes every step itself.
A step drawn from another proposal q — a smaller model whose tokens the
large one only scores, a learned proposal — returns
(steer/weighted state log-w) with log-w = log p − log q of the step; the
weight joins the step's factor and the target is still p·exp(reward).
Each particle's trace holds its states ([:steer/state t]) and reward
(:steer/reward); :record ((fn [state]), default the state) chooses
what is kept of a state — a language model's tokens rather than its KV
cache. foerster.learn/trajectories reads them back with the
particles' normalized weights, and learn/draws resamples them into
unweighted draws from the target: training data for the value estimates and
proposals that make the next search cheaper.
When a static parameter is conjugate to its data, the program can carry its
posterior instead of a sampled value (org.replikativ.foerster.conjugate):
score each datum under the posterior predictive, update the posterior in
closed form, and draw the parameter only if it is needed:
(spin (loop [c (conjugate/beta-bernoulli 1 1) [y & more] ys]
(if y
(do (observe (conjugate/predictive c) y)
(recur (conjugate/update c y) more))
(sample (conjugate/posterior c) :id :p))))
That is exact marginalization (Rao-Blackwellization; the conjugate case of
delayed sampling, Murray et al. 2018): under SMC the parameter never
degenerates, because it is never resampled, and streaming pays the same per
observation however long the stream. Families: normal-mean (known
observation sd), beta-bernoulli, gamma-poisson (predictive: the negative
binomial) and dirichlet-discrete. A parameter that is not conjugate to its
data needs PMMH or SMC² (below).
A state-space program with parameters θ — named sample sites — is filtered
best by SMC once θ is known; org.replikativ.foerster.smc2 infers θ by
running SMC conditioned on it. With the parameter sites constrained, SMC's
evidence estimate is p(θ)·p̂(y | θ), so two parameter values compare by the
difference of their SMC log-evidences.
;; a Markov chain over θ, each proposal scored by an SMC of 100 particles
(smc2/pmmh (model) 100 2000 {:params #{:sigma} :scale 0.2 :burn 200})
;; online, for a program whose data arrive at stream sites
(smc2/smc2 (model) {:params #{:sigma} :n-theta 50 :n-x 100})
;; → a step like smc/stream's: :push the next observation, :measure over θ
:thetas is
the chain after burn-in, :acceptance the rate over every iteration.:ess-target, they are resampled — a duplicate forks its inner
population's worlds (smc/stream's :fork), so copies evolve
independently — and moved by PMMH steps that run a fresh inner filter on
the data so far. m/log-marginal of its measure is the evidence of the
data so far, observations the program makes before its first stream site
included.θ's prior is the program's own: a θ-particle starts from a simulation of the
program. A proposal that leaves the support — a distribution refusing its
parameters — has density zero and is rejected; one outside the prior's
support is rejected too, after its SMC ran. The default random walk moves
continuous parameters; give discrete ones a :propose of their own. The
caller's :policy (proposals, other constraints) carries into every inner
SMC. Correlated PMMH, which needs SMC driven by explicit noise, is not
provided.
(kernel-infer model kernel n opts) runs a kernel from
org.replikativ.foerster.kernel (k):
| Kernel | Runs |
|---|---|
(k/prior-kernel) | SMC from the prior (:barrier-policy :none: importance sampling) |
a custom PInferenceKernel | SMC whose latent sites take the kernel's step value (extending) |
(k/single-site-mh-kernel iterations opts) | n independent Markov chains of single-site MH, proposals from the prior |
(k/random-walk-mh-kernel iterations {:step-size 0.1}) | random-walk MH on continuous sites |
(k/block-gibbs-kernel iterations selector block-kernels classifier opts) | block Gibbs: classifier (fn [address entry]) assigns sites to block ids, selector (k/round-robin-selector, k/random-selector) picks a block per iteration, block-kernels {block-id kernel} moves it (k/prior-block-kernel, k/random-walk-block-kernel) |
(k/hmc-kernel iterations {:step-size 0.1 :steps 10}) | HMC-within-Gibbs on block sites (blocks) |
(k/cycle iterations [k₁ k₂ …]) | each iteration runs k₁'s moves, then k₂'s, … — the composition K₂∘K₁ |
(k/mixture iterations [[w₁ k₁] [w₂ k₂] …]) | each iteration runs one kᵢ, picked with probability ∝ wᵢ |
Kernels that leave the posterior invariant form a monoid under composition,
and their mixtures with state-independent weights are invariant too: cycle
and mixture build a sampler from parts — HMC for the continuous block,
single-site moves for the discrete sites — and nest. Each part runs for its
own iterations.
The Markov-chain kernels run in fresh worlds only (they refuse :fork) and
take output options: :samples :final (default) emits each chain's last
state; :samples :all emits every state after the first :burn moves,
equally weighted.
(smc/stream model n opts) (org.replikativ.foerster.smc) is a CPS
operation resolving a step {:measure :push :fork :done? :close} once every
particle waits at its next :stream site; ((:push step) value) scores the
value and runs on (streaming notebook).
((:fork step)) resolves an independent copy of the population — every
waiting particle's world forked — to be pushed and closed on its own.
org.replikativ.foerster.gfi: simulate, generate, assess, update,
regenerate, mh, close! — CPS operations on traces of a model spin.org.replikativ.foerster.involutive/step: an involutive MCMC move.(counterfactual/counterfactual model {:evidence :interventions :particles}): twin-world counterfactuals of the model spin.See the programmable and counterfactuals notebooks.
Can you improve this documentation?Edit on GitHub
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |