# Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data

arXiv: https://arxiv.org/abs/1412.4869 · HTML: https://mlsanity.com/paper/1412.4869/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2014.

## Built on by (most cited first, 14 of 14)

- [DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs](https://mlsanity.com/paper/2106.06300/) (2021)
- [Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs](https://mlsanity.com/paper/2403.13748/) (2024)
- [Knowledge Adaptation as Posterior Correction](https://mlsanity.com/paper/2506.14262/) (2025)
- [Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry](https://mlsanity.com/paper/2511.01064/) (2025)
- [Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm](https://mlsanity.com/paper/2209.08139/) (2022)
- [Entropic Matching for Expectation Propagation of Markov Jump Processes](https://mlsanity.com/paper/2309.15604/) (2023)
- [Gaussian Ensemble Belief Propagation for Efficient Inference in High-Dimensional Systems](https://mlsanity.com/paper/2402.08193/) (2024)
- [Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework](https://mlsanity.com/paper/2502.00846/) (2025)
- [Expectation propagation for the smoothing distribution in dynamic probit](https://mlsanity.com/paper/2309.01641/) (2023)
- [Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression](https://mlsanity.com/paper/2309.08783/) (2023)
- [Sparse high-dimensional linear mixed modeling with a partitioned empirical Bayes ECM algorithm](https://mlsanity.com/paper/2310.12285/) (2023)
- [Fearless Stochasticity in Expectation Propagation](https://mlsanity.com/paper/2406.01801/) (2024)
- [Approximate Bayesian inference for cumulative probit regression models](https://mlsanity.com/paper/2511.06967/) (2025)
- [Neural composite likelihood estimation: simulation based inference for time series](https://mlsanity.com/paper/2609.21762/) (2026)

## Abstract (arXiv)

A common divide-and-conquer approach for Bayesian computation with big data is to partition the data, perform local inference for each piece separately, and combine the results to obtain a global posterior approximation. While being conceptually and computationally appealing, this method involves the problematic need to also split the prior for the local inferences; these weakened priors may not provide enough regularization for each separate computation, thus eliminating one of the key advantages of Bayesian methods. To resolve this dilemma while still retaining the generalizability of the underlying local inference method, we apply the idea of expectation propagation (EP) as a framework for distributed Bayesian inference. The central idea is to iteratively update approximations to the local likelihoods given the state of the other approximations and the prior. The present paper has two roles: we review the steps that are needed to keep EP algorithms numerically stable, and we suggest a general approach, inspired by EP, for approaching data partitioning problems in a way that achieves the computational benefits of parallelism while allowing each local update to make use of relevant information from the other sites. In addition, we demonstrate how the method can be applied in a hierarchical context to make use of partitioning of both data and parameters. The paper describes a general algorithmic framework, rather than a specific algorithm, and presents an example implementation for it.