# Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networks

arXiv: https://arxiv.org/abs/2502.06335 · HTML: https://mlsanity.com/paper/2502.06335/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 56 references that are in the map.
- First posted on arXiv 10 Feb 2025.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [Subspace Inference for Bayesian Deep Learning](https://mlsanity.com/paper/1907.07504/) (2019)
- [Rank-normalization, folding, and localization: An improved widehatR for assessing convergence of MCMC](https://mlsanity.com/paper/1903.08008/) (2019)
- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting](https://mlsanity.com/paper/2010.06772/) (2020)
- [Understanding predictive information criteria for Bayesian models](https://mlsanity.com/paper/1307.5928/) (2013)
- [Challenges in Markov chain Monte Carlo for Bayesian neural networks](https://mlsanity.com/paper/1910.06539/) (2019)
- [Stochastic Localization via Iterative Posterior Sampling](https://mlsanity.com/paper/2402.10758/) (2024)
- [Liouville Flow Importance Sampler](https://mlsanity.com/paper/2405.06672/) (2024)
- [Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetry](https://mlsanity.com/paper/2304.02902/) (2023)
- [A Bayesian neural network predicts the dissolution of compact planetary systems](https://mlsanity.com/paper/2101.04117/) (2021)
- [Asymptotic bias of inexact Markov Chain Monte Carlo methods in high dimension](https://mlsanity.com/paper/2108.00682/) (2021)
- [Path-Guided Particle-based Sampling](https://mlsanity.com/paper/2412.03312/) (2024)
- [Fluctuation without dissipation: Microcanonical Langevin Monte Carlo](https://mlsanity.com/paper/2303.18221/) (2023)

## Built on by (most cited first, 10 of 10)

- [Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?](https://mlsanity.com/paper/2602.06500/) (2026)
- [Paths and Ambient Spaces in Neural Loss Landscapes](https://mlsanity.com/paper/2503.03382/) (2025)
- [Symmetry Breaking in Transformers for Efficient and Interpretable Training](https://mlsanity.com/paper/2601.22257/) (2026)
- [On the Epistemic Uncertainty of Overparametrized Neural Networks](https://mlsanity.com/paper/2605.25234/) (2026)
- [Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles](https://mlsanity.com/paper/2505.13585/) (2025)
- [bde: A Python Package for Bayesian Deep Ensembles via MILE](https://mlsanity.com/paper/2605.14146/) (2026)
- [Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning](https://mlsanity.com/paper/2605.21765/) (2026)
- [Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification](https://mlsanity.com/paper/2605.12208/) (2026)
- [Uncertainty quantification for trustworthy deep learning: Methods and measures](https://mlsanity.com/paper/2607.28248/) (2026)
- [Microcanonical Hamiltonian Monte Carlo and the Helmholtz Theorem](https://mlsanity.com/paper/2609.07620/) (2026)

## Abstract (arXiv)

Despite recent advances, sampling-based inference for Bayesian Neural Networks (BNNs) remains a significant challenge in probabilistic deep learning. While sampling-based approaches do not require a variational distribution assumption, current state-of-the-art samplers still struggle to navigate the complex and highly multimodal posteriors of BNNs. As a consequence, sampling still requires considerably longer inference times than non-Bayesian methods even for small neural networks, despite recent advances in making software implementations more efficient. Besides the difficulty of finding high-probability regions, the time until samplers provide sufficient exploration of these areas remains unpredictable. To tackle these challenges, we introduce an ensembling approach that leverages strategies from optimization and a recently proposed sampler called Microcanonical Langevin Monte Carlo (MCLMC) for efficient, robust and predictable sampling performance. Compared to approaches based on the state-of-the-art No-U-Turn Sampler, our approach delivers substantial speedups up to an order of magnitude, while maintaining or improving predictive performance and uncertainty quantification across diverse tasks and data modalities. The suggested Microcanonical Langevin Ensembles and modifications to MCLMC additionally enhance the method's predictability in resource requirements, facilitating easier parallelization. All in all, the proposed method offers a promising direction for practical, scalable inference for BNNs.