# Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Drifts

arXiv: https://arxiv.org/abs/2009.09535 · HTML: https://mlsanity.com/paper/2009.09535/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2020.

## Built on by (most cited first, 12 of 12)

- [Sample-dependent Adaptive Temperature Scaling for Improved Calibration](https://mlsanity.com/paper/2207.06211/) (2022)
- [Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo](https://mlsanity.com/paper/2305.18246/) (2023)
- [Efficient and Generalizable Tuning Strategies for Stochastic Gradient MCMC](https://mlsanity.com/paper/2105.13059/) (2021)
- [Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metrics](https://mlsanity.com/paper/2303.05101/) (2023)
- [From Global to Local: A Scalable Benchmark for Local Posterior Sampling](https://mlsanity.com/paper/2507.21449/) (2025)
- [Training Latent Diffusion Models with Interacting Particle Algorithms](https://mlsanity.com/paper/2505.12412/) (2025)
- [Differentially private training of neural networks with Langevin dynamics for calibrated predictive uncertainty](https://mlsanity.com/paper/2107.04296/) (2021)
- [Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks](https://mlsanity.com/paper/2511.11666/) (2025)
- [Ito Diffusion Approximation of Universal Ito Chains for Sampling, Optimization and Boosting](https://mlsanity.com/paper/2310.06081/) (2023)
- [Humble your Overconfident Networks: Unlearning Overfitting via Sequential Monte Carlo Tempered Deep Ensembles](https://mlsanity.com/paper/2505.11671/) (2025)
- [Denoising-GS: Gaussian Splatting with Spatial-aware Denoising](https://mlsanity.com/paper/2605.14880/) (2026)
- [Generative Bayesian Filtering for State Estimation](https://mlsanity.com/paper/2607.20521/) (2026)

## Abstract (arXiv)

Bayesian deep learning offers a principled way to address many issues concerning safety of artificial intelligence (AI), such as model uncertainty,model interpretability, and prediction bias. However, due to the lack of efficient Monte Carlo algorithms for sampling from the posterior of deep neural networks (DNNs), Bayesian deep learning has not yet powered our AI system. We propose a class of adaptive stochastic gradient Markov chain Monte Carlo (SGMCMC) algorithms, where the drift function is biased to enhance escape from saddle points and the bias is adaptively adjusted according to the gradient of past samples. We establish the convergence of the proposed algorithms under mild conditions, and demonstrate via numerical examples that the proposed algorithms can significantly outperform the existing SGMCMC algorithms, such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian Monte Carlo (SGHMC) and preconditioned SGLD, in both simulation and optimization tasks.