# Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting

arXiv: https://arxiv.org/abs/2010.06772 · HTML: https://mlsanity.com/paper/2010.06772/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 30 of 54)

- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights](https://mlsanity.com/paper/2106.10185/) (2021)
- [Quantification of Uncertainty with Adversarial Models](https://mlsanity.com/paper/2307.03217/) (2023)
- [Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten](https://mlsanity.com/paper/2202.13585/) (2022)
- [Calibrating Multimodal Learning](https://mlsanity.com/paper/2306.01265/) (2023)
- [Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models](https://mlsanity.com/paper/2502.06999/) (2025)
- [Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamics](https://mlsanity.com/paper/2410.19780/) (2024)
- [Bayesian Deep Learning for Partial Differential Equation Parameter Discovery with Sparse and Noisy Data](https://mlsanity.com/paper/2108.04085/) (2021)
- [Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributions](https://mlsanity.com/paper/2409.20547/) (2024)
- [Revisiting Bayesian Autoencoders with MCMC](https://mlsanity.com/paper/2104.05915/) (2021)
- [Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networks](https://mlsanity.com/paper/2502.06335/) (2025)
- [Bayesian graph convolutional neural networks via tempered MCMC](https://mlsanity.com/paper/2104.08438/) (2021)
- [Training Bayesian Neural Networks with Sparse Subspace Variational Inference](https://mlsanity.com/paper/2402.11025/) (2024)
- [Diffusion and Flow-based Copulas: Forgetting and Remembering Dependencies](https://mlsanity.com/paper/2509.19707/) (2025)
- [Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks](https://mlsanity.com/paper/2106.09884/) (2021)
- [On Sequential Bayesian Inference for Continual Learning](https://mlsanity.com/paper/2301.01828/) (2023)
- [Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversion](https://mlsanity.com/paper/2303.07392/) (2023)
- [Entropy-MCMC: Sampling from Flat Basins with Ease](https://mlsanity.com/paper/2310.05401/) (2023)
- [Variational Inference of overparameterized Bayesian Neural Networks: a theoretical and empirical study](https://mlsanity.com/paper/2207.03859/) (2022)
- [CONFINE: Conformal Prediction for Interpretable Neural Networks](https://mlsanity.com/paper/2406.00539/) (2024)
- [Bayesian neural networks for predicting uncertainty in full-field material response](https://mlsanity.com/paper/2406.14838/) (2024)
- [Decentralized Bayesian Learning with Metropolis-Adjusted Hamiltonian Monte Carlo](https://mlsanity.com/paper/2107.07211/) (2021)
- [Gradient-free variational learning with conditional mixture networks](https://mlsanity.com/paper/2408.16429/) (2024)
- [Revisiting the Effects of Stochasticity for Hamiltonian Samplers](https://mlsanity.com/paper/2106.16200/) (2021)
- [Benchmarking Bayesian neural networks and evaluation metrics for regression tasks](https://mlsanity.com/paper/2206.06779/) (2022)
- [Normalizing Flows for Hierarchical Bayesian Analysis: A Gravitational Wave Population Study](https://mlsanity.com/paper/2211.09008/) (2022)
- [Correcting Auto-Differentiation in Neural-ODE Training](https://mlsanity.com/paper/2306.02192/) (2023)
- [L2M: Practical posterior Laplace approximation with optimization-driven second moment estimation](https://mlsanity.com/paper/2107.04695/) (2021)
- [QUBO Decision Tree: Annealing Machine Extends Decision Tree Splitting](https://mlsanity.com/paper/2303.09772/) (2023)

## Abstract (arXiv)

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) approach that exhibits favourable exploration properties in high-dimensional models such as neural networks. Unfortunately, HMC has limited use in large-data regimes and little work has explored suitable approaches that aim to preserve the entire Hamiltonian. In our work, we introduce a new symmetric integration scheme for split HMC that does not rely on stochastic gradients. We show that our new formulation is more efficient than previous approaches and is easy to implement with a single GPU. As a result, we are able to perform full HMC over common deep learning architectures using entire data sets. In addition, when we compare with stochastic gradient MCMC, we show that our method achieves better performance in both accuracy and uncertainty quantification. Our approach demonstrates HMC as a feasible option when considering inference schemes for large-scale machine learning problems.