# Information Geometry of Message Passing

arXiv: https://arxiv.org/abs/2608.15922 · HTML: https://mlsanity.com/paper/2608.15922/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 60 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 19 of 19)

- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Variational Learning is Effective for Large Deep Networks](https://mlsanity.com/paper/2402.17641/) (2024)
- [Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models](https://mlsanity.com/paper/1807.04489/) (2018)
- [Information Newton's flow: second-order optimization method in probability space](https://mlsanity.com/paper/2001.04341/) (2020)
- [Variational Message Passing with Structured Inference Networks](https://mlsanity.com/paper/1803.05589/) (2018)
- [Reactive Message Passing for Scalable Bayesian Inference](https://mlsanity.com/paper/2112.13251/) (2021)
- [Expected Free Energy-based Planning as Variational Inference](https://mlsanity.com/paper/2504.14898/) (2025)
- [Fast Approximate Inference for Arbitrarily Large Semiparametric Regression Models via Message Passing](https://mlsanity.com/paper/1602.07412/) (2016)
- [Bayesian Predictive Coding](https://mlsanity.com/paper/2503.24016/) (2025)
- [Information Geometry of Variational Bayes](https://mlsanity.com/paper/2509.15641/) (2025)
- [What Type of Inference is Active Inference?](https://mlsanity.com/paper/2606.04935/) (2026)
- [Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference](https://mlsanity.com/paper/2605.29467/) (2026)

## Abstract (arXiv)

We show that the natural-gradient stationary condition of variational inference has an edge-local form on a Forney-style factor graph. We start from the Bethe free energy and constrain a selected edge marginal to an exponential family. At a stationary point, the natural parameter of that edge equals the sum of two projected messages, one from each incident factor. Each projected message is the natural-gradient projection of the exact belief-propagation log-message at the current receiving marginal, or equivalently, the gradient of its expectation in the so-called mean coordinates. We call the resulting scheme natural-gradient message passing (NGMP). The rule is local; each edge may carry its own exponential family, and the message a factor sends depends on the marginal that receives it. Compared with variational message passing, NGMP keeps the part of the exact message that the receiving family can represent instead of averaging the factor under the neighboring beliefs. The two coincide when the uncertainty on the edges entering a non-conjugate factor vanishes, and NGMP is more accurate when that uncertainty persists, for example, along a partially observed latent chain or when parameters are filtered through successive data batches. Experiments on Poisson smoothing, heteroskedastic regression, and hourly ETTh forecasting confirm this and show that the gain appears mainly in uncertainty calibration.