# Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling

arXiv: https://arxiv.org/abs/1510.08692 · HTML: https://mlsanity.com/paper/1510.08692/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2015.

## Built on by (most cited first, 10 of 10)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021)
- [Quantifying Uncertainty in Deep Spatiotemporal Forecasting](https://mlsanity.com/paper/2105.11982/) (2021)
- [DeepGLEAM: A hybrid mechanistic and deep learning model for COVID-19 forecasting](https://mlsanity.com/paper/2102.06684/) (2021)
- [Revisiting the Effects of Stochasticity for Hamiltonian Samplers](https://mlsanity.com/paper/2106.16200/) (2021)
- [A Langevin sampling algorithm inspired by the Adam optimizer](https://mlsanity.com/paper/2504.18911/) (2025)
- [Adaptive Momentum and Nonlinear Damping for Neural Network Training](https://mlsanity.com/paper/2602.00334/) (2026)
- [Quantifying the mini-batching error in Bayesian inference for Adaptive Langevin dynamics](https://mlsanity.com/paper/2105.10347/) (2021)
- [Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks](https://mlsanity.com/paper/2511.11666/) (2025)
- [Improving the stability of the covariance-controlled adaptive Langevin thermostat for large-scale Bayesian sampling](https://mlsanity.com/paper/2512.24515/) (2025)

## Abstract (arXiv)

Monte Carlo sampling for Bayesian posterior inference is a common approach used in machine learning. The Markov Chain Monte Carlo procedures that are used are often discrete-time analogues of associated stochastic differential equations (SDEs). These SDEs are guaranteed to leave invariant the required posterior distribution. An area of current research addresses the computational benefits of stochastic gradient methods in this setting. Existing techniques rely on estimating the variance or covariance of the subsampling error, and typically assume constant variance. In this article, we propose a covariance-controlled adaptive Langevin thermostat that can effectively dissipate parameter-dependent noise while maintaining a desired target distribution. The proposed method achieves a substantial speedup over popular alternative schemes for large-scale machine learning applications.