# Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference

arXiv: https://arxiv.org/abs/1803.11159 · HTML: https://mlsanity.com/paper/1803.11159/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2018.

## Built on by (most cited first, 4 of 4)

- [Last Layer Hamiltonian Monte Carlo](https://mlsanity.com/paper/2507.08905/) (2025)
- [A New Framework for Variance-Reduced Hamiltonian Monte Carlo](https://mlsanity.com/paper/2102.04613/) (2021)
- [Target Detection on Hyperspectral Images Using MCMC and VI Trained Bayesian Neural Networks](https://mlsanity.com/paper/2308.06293/) (2023)
- [On Convergence of the Alternating Directions SGHMC Algorithm](https://mlsanity.com/paper/2405.13140/) (2024)

## Abstract (arXiv)

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stochastic gradients evaluated on mini-batches are used as a replacement. In order to reduce the high variance of noisy stochastic gradients, Dubey et al. [2016] applied the standard variance reduction technique on stochastic gradient Langevin dynamics and obtained both theoretical and experimental improvements. In this paper, we apply the variance reduction tricks on Hamiltonian Monte Carlo and achieve better theoretical convergence results compared with the variance-reduced Langevin dynamics. Moreover, we apply the symmetric splitting scheme in our variance-reduced Hamiltonian Monte Carlo algorithms to further improve the theoretical results. The experimental results are also consistent with the theoretical results. As our experiment shows, variance-reduced Hamiltonian Monte Carlo demonstrates better performance than variance-reduced Langevin dynamics in Bayesian regression and classification tasks on real-world datasets.