Deterministic nodes

Most nodes of RxInfer.jl are distributions, mainly from the exponential family, and compositions of them, such as the Gaussian controlled variance (GCV) and autoregressive (AR) nodes. A deterministic transformation of one or several random variables, y := f(x), is a node too: the delta node. Its messages have no closed form for an arbitrary f, so the node needs an approximation method, which you choose per node. This guide describes the methods and when each applies.

Features and supported inference scenarios

The delta node supports three approximation methods. Which one fits depends on the nodes around the delta node:

1. Gaussian nodes: for a delta node connected only to univariate or multivariate Gaussian distributions, use Linearization or Unscented.
2. Exponential family nodes: for a delta node connected to other members of the exponential family, use CVIProjection.
3. Stacked delta nodes: for delta nodes connected to each other, any of the three methods applies.
4. Inverse functions: when the inverse of f is known, Linearization and Unscented use it.

The node's algorithm is DeltaApproximation, which carries the method and, optionally, the inverse. You give it to the node with @algorithm, as Algorithm specification describes.

Gaussian case

For Gaussian distributions, use either Linearization or Unscented. Linearization is a first-order approximation. Unscented is a more precise second-order approximation, and it may need its hyperparameters tuned. Both methods work well for a differentiable function; for a function that is not differentiable, their results may be inaccurate.

Consider the following example:

using RxInfer

@model function delta_node_example(z)
    x  ~ Normal(mean = 0.0, var = 1.0)
    y := tanh(x)
    z  ~ Normal(mean = y, var = 1.0)
end

To run inference in this model, give the delta node, here the tanh function, its approximation method with @algorithm:

delta_algorithm = @algorithm begin
    tanh() -> DeltaApproximation(method = Linearization())
end

A method alone is a shorthand for DeltaApproximation(method = ...):

delta_algorithm = @algorithm begin
    tanh() -> Unscented()
end

The docstrings of Unscented and Linearization describe their parameters.

tanh is invertible, and giving its inverse lets the node compute the message towards x directly from the message from z:

delta_algorithm = @algorithm begin
    tanh() -> DeltaApproximation(method = Linearization(), inverse = atanh)
end

Pass the specification to infer as algorithm:

result = infer(
    model     = delta_node_example(),
    algorithm = delta_algorithm,
    data      = (z = 1.0,),
)

Inference results:
  Posteriors       | available for (y, x)

The same holds for a delta node with several inputs. For instance:

f(x, g) = x * tanh(g)

f (generic function with 1 method)

@model function delta_node_example(z)
    x ~ Normal(mean = 1.0, var = 1.0)
    g ~ Normal(mean = 1.0, var = 1.0)
    y := f(x, g)
    z ~ Normal(mean = y, var = 0.1)
end

The corresponding algorithm specification is

delta_algorithm = @algorithm begin
    f() -> DeltaApproximation(method = Linearization())
end

or, with the shorthand,

delta_algorithm = @algorithm begin
    f() -> Linearization()
end

result = infer(model = delta_node_example(), algorithm = delta_algorithm, data = (z = 1.0,))

Inference results:
  Posteriors       | available for (y, g, x)

When functions express each input of f in terms of the output and the other inputs, you can give them as a tuple of inverses, in the order of the inputs:

f_back_x(out, g) = out / tanh(g)
f_back_g(out, x) = atanh(out / x)

f_back_g (generic function with 1 method)

delta_algorithm = @algorithm begin
    f() -> DeltaApproximation(method = Linearization(), inverse = (f_back_x, f_back_g))
end

result = infer(model = delta_node_example(), algorithm = delta_algorithm, data = (z = 1.0,))

Inference results:
  Posteriors       | available for (y, g, x)

Exponential family case

When the delta node is connected to nodes of the exponential family other than Gaussians, Linearization and Unscented do not apply. CVIProjection does: it projects the node's messages onto members of the exponential family by stochastic optimization. Here is a modified example:

using RxInfer, ExponentialFamilyProjection, StableRNGs

@model function delta_node_example1(z)
    x ~ Gamma(shape = 1.0, rate = 1.0)
    y := tanh(x)
    z .~ Bernoulli(y)
end

CVIProjection projects onto the families that you name with ProjectedTo in @constraints, and its rules read the marginal of the output, which needs an initial value:

delta_algorithm = @algorithm begin
    tanh() -> DeltaApproximation(method = CVIProjection())
end

delta_constraints = @constraints begin
    q(x)::ProjectedTo(Gamma)
    q(y)::ProjectedTo(Beta)
end

delta_initialization = @initialization begin
    q(y) = Beta(1.0, 1.0)
end

CVIProjection samples. It draws from the random number generator of the engine, which you choose with the context option of infer, here for reproducible results:

result = infer(
    model          = delta_node_example1(),
    algorithm      = delta_algorithm,
    constraints    = delta_constraints,
    initialization = delta_initialization,
    data           = (z = [1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0],),
    iterations     = 10,
    options        = (context = (rng = StableRNG(42),),),
)

(x = result.posteriors[:x][end], y = result.posteriors[:y][end])

(x = Gamma{Float64}(α=7.62441664054263, θ=0.1282822177050464), y = Beta{Float64}(α=4.879050143710983, β=1.881449597729063))

The docstring of CVIProjection explains its hyperparameters. Also read the Non-conjugate Inference section.

Fuse deterministic nodes with stochastic nodes

You can also avoid the approximation altogether, by fusing the deterministic relation with a neighboring stochastic node, as this section shows.