# Expectation Propagation for approximate Bayesian inference

arXiv: https://arxiv.org/abs/1301.2294 · HTML: https://mlsanity.com/paper/1301.2294/

## Key facts

- Cited by 174 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2013.

## Built on by (most cited first, 30 of 174)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022)
- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization](https://mlsanity.com/paper/2310.02679/) (2023)
- [Joint Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/2210.02905/) (2022)
- [Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models](https://mlsanity.com/paper/2309.11420/) (2023)
- [Graph-based Approximate Message Passing Iterations](https://mlsanity.com/paper/2109.11905/) (2021)
- [Is MC Dropout Bayesian?](https://mlsanity.com/paper/2110.04286/) (2021)
- [Autoregressive Conditional Neural Processes](https://mlsanity.com/paper/2303.14468/) (2023)
- [Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial](https://mlsanity.com/paper/2305.04933/) (2023)
- [Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound](https://mlsanity.com/paper/2102.09788/) (2021)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes](https://mlsanity.com/paper/2302.01513/) (2023)
- [On the Convergence of Coordinate Ascent Variational Inference](https://mlsanity.com/paper/2306.01122/) (2023)
- [A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classification](https://mlsanity.com/paper/2109.11301/) (2021)
- [Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach](https://mlsanity.com/paper/2302.04228/) (2023)
- [Variational Inference with Gaussian Score Matching](https://mlsanity.com/paper/2307.07849/) (2023)
- [Amortized Variational Inference: A Systematic Review](https://mlsanity.com/paper/2209.10888/) (2022)
- [Solving Common-Payoff Games with Approximate Policy Iteration](https://mlsanity.com/paper/2101.04237/) (2021)
- [Amortized In-Context Bayesian Posterior Estimation](https://mlsanity.com/paper/2502.06601/) (2025)
- [A visual introduction to Gaussian Belief Propagation](https://mlsanity.com/paper/2107.02308/) (2021)
- [Conditional Variational Autoencoder for Learned Image Reconstruction](https://mlsanity.com/paper/2110.11681/) (2021)
- [Partitioned Variational Inference: A Framework for Probabilistic Federated Learning](https://mlsanity.com/paper/2202.12275/) (2022)
- [Conditional Injective Flows for Bayesian Imaging](https://mlsanity.com/paper/2204.07664/) (2022)
- [Multi-Objective Bayesian Optimization with Active Preference Learning](https://mlsanity.com/paper/2311.13460/) (2023)
- [Variational Bayesian Optimal Experimental Design with Normalizing Flows](https://mlsanity.com/paper/2404.13056/) (2024)
- [Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptation](https://mlsanity.com/paper/2502.02463/) (2025)
- [Deep learning via message passing algorithms based on belief propagation](https://mlsanity.com/paper/2110.14583/) (2021)
- [Safe Active Learning for Time-Series Modeling with Gaussian Processes](https://mlsanity.com/paper/2402.06276/) (2024)

## Abstract (arXiv)

This paper presents a new deterministic approximation technique in Bayesian networks. This method, "Expectation Propagation", unifies two previous techniques: assumed-density filtering, an extension of the Kalman filter, and loopy belief propagation, an extension of belief propagation in Bayesian networks. All three algorithms try to recover an approximate distribution which is close in KL divergence to the true distribution. Loopy belief propagation, because it propagates exact belief states, is useful for a limited class of belief networks, such as those which are purely discrete. Expectation Propagation approximates the belief states by only retaining certain expectations, such as mean and variance, and iterates until these expectations are consistent throughout the network. This makes it applicable to hybrid networks with discrete and continuous nodes. Expectation Propagation also extends belief propagation in the opposite direction - it can propagate richer belief states that incorporate correlations between nodes. Experiments with Gaussian mixture models show Expectation Propagation to be convincingly better than methods with similar computational cost: Laplace's method, variational Bayes, and Monte Carlo. Expectation Propagation also provides an efficient algorithm for training Bayes point machine classifiers.