# Learning in Deep Factor Graphs with Gaussian Belief Propagation

arXiv: https://arxiv.org/abs/2311.14649 · HTML: https://mlsanity.com/paper/2311.14649/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 24 Nov 2023.

## Builds on (most cited first, 19 of 19)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers](https://mlsanity.com/paper/1811.04918/) (2018)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Loopy Belief Propagation for Approximate Inference: An Empirical Study](https://mlsanity.com/paper/1301.6725/) (2013)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Learning to Decode Linear Codes Using Deep Learning](https://mlsanity.com/paper/1607.04793/) (2016)
- [Neural Enhanced Belief Propagation on Factor Graphs](https://mlsanity.com/paper/2003.01998/) (2020)
- [Inference in Probabilistic Graphical Models by Graph Neural Networks](https://mlsanity.com/paper/1803.07710/) (2018)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [FutureMapping 2: Gaussian Belief Propagation for Spatial AI](https://mlsanity.com/paper/1910.14139/) (2019)
- [A Theoretical Framework for Inference Learning](https://mlsanity.com/paper/2206.00164/) (2022)
- [Bundle Adjustment on a Graph Processor](https://mlsanity.com/paper/2003.03134/) (2020)
- [A Berkeley View of Systems Challenges for AI](https://mlsanity.com/paper/1712.05855/) (2017)
- [A visual introduction to Gaussian Belief Propagation](https://mlsanity.com/paper/2107.02308/) (2021)
- [Deep learning via message passing algorithms based on belief propagation](https://mlsanity.com/paper/2110.14583/) (2021)
- [A Robot Web for Distributed Many-Device Localisation](https://mlsanity.com/paper/2202.03314/) (2022)
- [Hierarchical compositional feature learning](https://mlsanity.com/paper/1611.02252/) (2016)
- [Distributing Collaborative Multi-Robot Planning with Gaussian Belief Propagation](https://mlsanity.com/paper/2203.11618/) (2022)

## Built on by (most cited first, 1 of 1)

- [PixVOD: Pixel-Distributed Direct Visual Odometry and Depth Estimation](https://mlsanity.com/paper/2606.03989/) (2026)

## Abstract (arXiv)

We propose an approach to do learning in Gaussian factor graphs. We treat all relevant quantities (inputs, outputs, parameters, latents) as random variables in a graphical model, and view both training and prediction as inference problems with different observed nodes. Our experiments show that these problems can be efficiently solved with belief propagation (BP), whose updates are inherently local, presenting exciting opportunities for distributed and asynchronous training. Our approach can be scaled to deep networks and provides a natural means to do continual learning: use the BP-estimated parameter marginals of the current task as parameter priors for the next. On a video denoising task we demonstrate the benefit of learnable parameters over a classical factor graph approach and we show encouraging performance of deep factor graphs for continual image classification.