# Adaptively Preconditioned Stochastic Gradient Langevin Dynamics

arXiv: https://arxiv.org/abs/1906.04324 · HTML: https://mlsanity.com/paper/1906.04324/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 5 of 5)

- [Langevin algorithms for Markovian Neural Networks and Deep Stochastic control](https://mlsanity.com/paper/2212.12018/) (2022)
- [Langevin algorithms for very deep Neural Networks with application to image classification](https://mlsanity.com/paper/2212.14718/) (2022)
- [Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments](https://mlsanity.com/paper/2506.21129/) (2025)
- [Ito Diffusion Approximation of Universal Ito Chains for Sampling, Optimization and Boosting](https://mlsanity.com/paper/2310.06081/) (2023)
- [Meta Policy Switching for Secure UAV Deconfliction in Adversarial Airspace](https://mlsanity.com/paper/2506.21127/) (2025)

## Abstract (arXiv)

Stochastic Gradient Langevin Dynamics infuses isotropic gradient noise to SGD to help navigate pathological curvature in the loss landscape for deep networks. Isotropic nature of the noise leads to poor scaling, and adaptive methods based on higher order curvature information such as Fisher Scoring have been proposed to precondition the noise in order to achieve better convergence. In this paper, we describe an adaptive method to estimate the parameters of the noise and conduct experiments on well-known model architectures to show that the adaptively preconditioned SGLD method achieves convergence with the speed of adaptive first order methods such as Adam, AdaGrad etc. and achieves generalization equivalent of SGD in the test set.