Numerical blocks as choice sites (the spindel side of the spindel ↔ raster block contract, spindel-raster doc/contract.md).
A block is a fixed-shape group of latent variables with the density factors they touch, given as a description (data) and capabilities (functions over primitive arrays):
(block {:block/id :gauss :block/latents [{:name :mu :shape [2] :support :real}] :block/target :complete-conditional} {:log-density (fn [^doubles theta inputs] lp) :value+grad (fn [^doubles theta inputs] [lp ^doubles grad])})
(block-dist b inputs) is the block at its inputs as a distribution whose
density is the block's log target, so a block site is an ordinary choice
site: (sample (block-dist b inputs) :id :mu :init [0.0 0.0]). Its value is
θ, the latents flattened in declared order, as a vector of doubles.
The target includes every factor the latents touch: their priors and the
observations that depend on them, which must then not be observed again as
sites of their own. A block without :sample cannot be drawn from; start
it at an :init. :sample need not draw from the target (it cannot know
its normalizer); a block drawn from under importance sampling or SMC, or
proposed from by single-site MH, also needs :sample-log-density, the log
density of what :sample draws, which weighs the draw:
log target(θ) − log sample-density(θ).
Constrained latents need the block to declare :block/coordinates :constrained and give its capabilities in those natural coordinates (σ
itself, p itself): block then works in unconstrained θ, adding each
transform's log-Jacobian to the density and the chain rule to the
gradient, so HMC and NUTS move freely. The supports, with their
transforms (Stan's and PyMC's):
:real x = θ :positive x = exp θ [:lower a] x = a + exp θ [:upper b] x = b − exp θ [:interval a b] x = a + (b − a)·logistic(θ) :ordered a vector [K], increasing: x₀ = θ₀, xₖ = xₖ₋₁ + exp θₖ :simplex a vector [K] of probabilities summing to 1, from K − 1 coordinates of θ by stick-breaking
An unknown support, or one that does not fit its latent's shape, is
refused when the block is defined (::unknown-support). (A Cholesky
factor of a correlation matrix is not among them yet.) θ is what the
trace holds; constrain maps it to the latents' values and unconstrain
back. Without the declaration, a block's capabilities are in
unconstrained coordinates already and every latent must be :real.
The observations inside a block's target are invisible to the trace. Two
optional capabilities show them: :pointwise (fn [theta inputs]) →
{key log-lik}, each observation's log density, and :simulate (fn [theta
inputs]) → {key value}, a fresh draw of each from its law. With them
diagnostics/pointwise-log-likelihood (hence LOO and WAIC) and
infer/predictive see the block's observations, at [site-address key].
Numerical blocks as choice sites (the spindel side of the spindel ↔ raster
block contract, spindel-raster doc/contract.md).
A block is a fixed-shape group of latent variables with the density factors
they touch, given as a description (data) and capabilities (functions over
primitive arrays):
(block {:block/id :gauss
:block/latents [{:name :mu :shape [2] :support :real}]
:block/target :complete-conditional}
{:log-density (fn [^doubles theta inputs] lp)
:value+grad (fn [^doubles theta inputs] [lp ^doubles grad])})
`(block-dist b inputs)` is the block at its inputs as a distribution whose
density is the block's log target, so a block site is an ordinary choice
site: `(sample (block-dist b inputs) :id :mu :init [0.0 0.0])`. Its value is
θ, the latents flattened in declared order, as a vector of doubles.
The target includes every factor the latents touch: their priors and the
observations that depend on them, which must then not be observed again as
sites of their own. A block without `:sample` cannot be drawn from; start
it at an `:init`. `:sample` need not draw from the target (it cannot know
its normalizer); a block drawn from under importance sampling or SMC, or
proposed from by single-site MH, also needs `:sample-log-density`, the log
density of what `:sample` draws, which weighs the draw:
log target(θ) − log sample-density(θ).
Constrained latents need the block to declare `:block/coordinates
:constrained` and give its capabilities in those natural coordinates (σ
itself, p itself): `block` then works in unconstrained θ, adding each
transform's log-Jacobian to the density and the chain rule to the
gradient, so HMC and NUTS move freely. The supports, with their
transforms (Stan's and PyMC's):
:real x = θ
:positive x = exp θ
[:lower a] x = a + exp θ
[:upper b] x = b − exp θ
[:interval a b] x = a + (b − a)·logistic(θ)
:ordered a vector [K], increasing: x₀ = θ₀, xₖ = xₖ₋₁ + exp θₖ
:simplex a vector [K] of probabilities summing to 1, from K − 1
coordinates of θ by stick-breaking
An unknown support, or one that does not fit its latent's shape, is
refused when the block is defined (`::unknown-support`). (A Cholesky
factor of a correlation matrix is not among them yet.) θ is what the
trace holds; `constrain` maps it to the latents' values and `unconstrain`
back. Without the declaration, a block's capabilities are in
unconstrained coordinates already and every latent must be `:real`.
The observations inside a block's target are invisible to the trace. Two
optional capabilities show them: `:pointwise` (fn [theta inputs]) →
{key log-lik}, each observation's log density, and `:simulate` (fn [theta
inputs]) → {key value}, a fresh draw of each from its law. With them
`diagnostics/pointwise-log-likelihood` (hence LOO and WAIC) and
`infer/predictive` see the block's observations, at [site-address key].(block description capabilities)A block from its description and capabilities (see the namespace).
A block from its `description` and `capabilities` (see the namespace).
(block-dist b inputs)Block b at inputs, as the distribution of its site.
Block `b` at `inputs`, as the distribution of its site.
(capability b k)The capability k of block b, or nil.
The capability `k` of block `b`, or nil.
(constrain b theta)The latents' values at θ: θ mapped back through each latent's transform
(identity for :real), as a vector in the latents' declared order (a
:simplex of K takes K − 1 coordinates of θ and gives K).
The latents' values at θ: θ mapped back through each latent's transform (identity for `:real`), as a vector in the latents' declared order (a `:simplex` of K takes K − 1 coordinates of θ and gives K).
(jacobian b theta)[log |det dx/dθ|, its gradient in θ] of block b's transforms at θ: the
term a constrained block adds to its density (0 for an unconstrained
block). Subtract it for the mode in the latents' own coordinates.
[log |det dx/dθ|, its gradient in θ] of block `b`'s transforms at θ: the term a constrained block adds to its density (0 for an unconstrained block). Subtract it for the mode in the latents' own coordinates.
(latent b theta name)The latent name of the latents' values x (constrained θ; θ itself
for an unconstrained block): a double for a scalar, a vector otherwise.
The latent `name` of the latents' values `x` (`constrain`ed θ; θ itself for an unconstrained block): a double for a scalar, a vector otherwise.
(unconstrain b x)θ of the latents' values x (a vector in their declared order): the
inverse of constrain.
θ of the latents' values `x` (a vector in their declared order): the inverse of `constrain`.
(value+grad dist theta)[log target, gradient] of dist (a block-dist) at θ, both from the
block's :value+grad; the gradient as a vector.
[log target, gradient] of `dist` (a `block-dist`) at θ, both from the block's `:value+grad`; the gradient as a vector.
(value+grad! dist theta)As value+grad on a double array θ, the gradient a double array: for
samplers that keep their state in arrays. θ is not modified.
As `value+grad` on a double array θ, the gradient a double array: for samplers that keep their state in arrays. θ is not modified.
(with-numeric-gradient capabilities)(with-numeric-gradient {:keys [log-density] :as capabilities}
{:keys [h] :or {h 1.0E-6}})capabilities with a :value+grad computed from :log-density by
central differences of step h (default 1e-6 times the coordinate's
scale): 2n evaluations per gradient, accurate to about 1e-8 relative on a
smooth density. An explicit opt-in for small blocks written in plain
Clojure — foerster never differentiates numerically behind your back; for
a compiled gradient, write the block with foerster-raster.
`capabilities` with a `:value+grad` computed from `:log-density` by central differences of step `h` (default 1e-6 times the coordinate's scale): 2n evaluations per gradient, accurate to about 1e-8 relative on a smooth density. An explicit opt-in for small blocks written in plain Clojure — foerster never differentiates numerically behind your back; for a compiled gradient, write the block with foerster-raster.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |