# Patterns of Scalable Bayesian Inference

arXiv: https://arxiv.org/abs/1602.05221 · HTML: https://mlsanity.com/paper/1602.05221/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2016.

## Built on by (most cited first, 25 of 25)

- [Pathfinder: Parallel quasi-Newton variational inference](https://mlsanity.com/paper/2108.03782/) (2021)
- [FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation](https://mlsanity.com/paper/2310.00339/) (2023)
- [Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction](https://mlsanity.com/paper/2210.03067/) (2022)
- [Bayesian Continual Learning via Spiking Neural Networks](https://mlsanity.com/paper/2208.13723/) (2022)
- [Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling](https://mlsanity.com/paper/2205.14283/) (2022)
- [Parallel MCMC Without Embarrassing Failures](https://mlsanity.com/paper/2202.11154/) (2022)
- [Client Selection for Federated Bayesian Learning](https://mlsanity.com/paper/2212.05492/) (2022)
- [Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMC](https://mlsanity.com/paper/2104.03942/) (2021)
- [Interpolating between sampling and variational inference with infinite stochastic mixtures](https://mlsanity.com/paper/2110.09618/) (2021)
- [Spatiotemporal Clustering with Neyman-Scott Processes via Connections to Bayesian Nonparametric Mixture Models](https://mlsanity.com/paper/2201.05044/) (2022)
- [Leveraging Channel Noise for Sampling and Privacy via Quantized Federated Langevin Monte Carlo](https://mlsanity.com/paper/2202.13932/) (2022)
- [Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges](https://mlsanity.com/paper/2211.14732/) (2022)
- [Calibration-Aware Bayesian Learning](https://mlsanity.com/paper/2305.07504/) (2023)
- [Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalization](https://mlsanity.com/paper/2108.00785/) (2021)
- [Federated Learning with Uncertainty and Personalization via Efficient Second-order Optimization](https://mlsanity.com/paper/2411.18385/) (2024)
- [Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective](https://mlsanity.com/paper/2512.05267/) (2025)
- [Bayesian Active Meta-Learning for Black-Box Optimization](https://mlsanity.com/paper/2110.09943/) (2021)
- [Comparison of parallel SMC and MCMC for Bayesian deep learning](https://mlsanity.com/paper/2402.06173/) (2024)
- [Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles](https://mlsanity.com/paper/2505.13585/) (2025)
- [To Trust or Not to Trust: On Calibration in ML-based Resource Allocation for Wireless Networks](https://mlsanity.com/paper/2507.17494/) (2025)
- [Robust Distributed Bayesian Learning with Stragglers via Consensus Monte Carlo](https://mlsanity.com/paper/2112.09794/) (2021)
- [Variational Hierarchical Mixtures for Probabilistic Learning of Inverse Dynamics](https://mlsanity.com/paper/2211.01120/) (2022)
- [Calibrating AI Models for Wireless Communications via Conformal Prediction](https://mlsanity.com/paper/2212.07775/) (2022)
- [Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme](https://mlsanity.com/paper/2408.12888/) (2024)
- [A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment](https://mlsanity.com/paper/2607.17855/) (2026)

## Abstract (arXiv)

Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge. In response to this challenge, there has been considerable recent work based on varying assumptions about model structure, underlying computational resources, and the importance of asymptotic correctness. As a result, there is a zoo of ideas with few clear overarching principles. In this paper, we seek to identify unifying principles, patterns, and intuitions for scaling Bayesian inference. We review existing work on utilizing modern computing resources with both MCMC and variational approximation techniques. From this taxonomy of ideas, we characterize the general principles that have proven successful for designing scalable inference procedures and comment on the path forward.