# Stochastic Expectation Propagation

arXiv: https://arxiv.org/abs/1506.04132 · HTML: https://mlsanity.com/paper/1506.04132/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2015.

## Built on by (most cited first, 22 of 22)

- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach](https://mlsanity.com/paper/2302.04228/) (2023)
- [Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees](https://mlsanity.com/paper/2111.01721/) (2021)
- [Partitioned Variational Inference: A Framework for Probabilistic Federated Learning](https://mlsanity.com/paper/2202.12275/) (2022)
- [Sparse Algorithms for Markovian Gaussian Processes](https://mlsanity.com/paper/2103.10710/) (2021)
- [Post-hoc loss-calibration for Bayesian neural networks](https://mlsanity.com/paper/2106.06997/) (2021)
- [Task Agnostic and Post-hoc Unseen Distribution Detection](https://mlsanity.com/paper/2207.13083/) (2022)
- [Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Models](https://mlsanity.com/paper/2306.04201/) (2023)
- [VIKING: Deep variational inference with stochastic projections](https://mlsanity.com/paper/2510.23684/) (2025)
- [Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise](https://mlsanity.com/paper/2103.08497/) (2021)
- [Differentially private partitioned variational inference](https://mlsanity.com/paper/2209.11595/) (2022)
- [Differentially private stochastic expectation propagation (DP-SEP)](https://mlsanity.com/paper/2111.13219/) (2021)
- [Improving Uncertainty Quantification of Variance Networks by Tree-Structured Learning](https://mlsanity.com/paper/2212.12658/) (2022)
- [Direct Message Approximation (DMA): A Consistency-Based Framework for Tractable Approximate Inference on Factor Graphs](https://mlsanity.com/paper/2609.29466/) (2026)
- [Approximate Inference via Clustering](https://mlsanity.com/paper/2111.14219/) (2021)
- [Image Reconstruction by Splitting Expectation Propagation Techniques from Iterative Inversion](https://mlsanity.com/paper/2208.12340/) (2022)
- [Variational Inference on the Final-Layer Output of Neural Networks](https://mlsanity.com/paper/2302.02420/) (2023)
- [Fearless Stochasticity in Expectation Propagation](https://mlsanity.com/paper/2406.01801/) (2024)
- [Training of Spiking Neural Networks with Expectation-Propagation](https://mlsanity.com/paper/2506.23757/) (2025)
- [Sparse Gaussian Processes: Structured Approximations and Power-EP Revisited](https://mlsanity.com/paper/2507.02377/) (2025)

## Abstract (arXiv)

Expectation propagation (EP) is a deterministic approximation algorithm that is often used to perform approximate Bayesian parameter learning. EP approximates the full intractable posterior distribution through a set of local approximations that are iteratively refined for each datapoint. EP can offer analytic and computational advantages over other approximations, such as Variational Inference (VI), and is the method of choice for a number of models. The local nature of EP appears to make it an ideal candidate for performing Bayesian learning on large models in large-scale dataset settings. However, EP has a crucial limitation in this context: the number of approximating factors needs to increase with the number of data-points, N, which often entails a prohibitively large memory overhead. This paper presents an extension to EP, called stochastic expectation propagation (SEP), that maintains a global posterior approximation (like VI) but updates it in a local way (like EP). Experiments on a number of canonical learning problems using synthetic and real-world datasets indicate that SEP performs almost as well as full EP, but reduces the memory consumption by a factor of N. SEP is therefore ideally suited to performing approximate Bayesian learning in the large model, large dataset setting.