Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation
Not on the map. Why?
Abstract (arXiv)
Stochastic-gradient MCMC methods enable scalable Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose Stochastic Gradient Lattice Random Walk (SGLRW), an extension of the Lattice Random Walk discretization. Unlike conventional Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as comparison we analyze a natural analogue of SGLD utilizing gradient clipping. Experimental validation on Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 17 Feb 2026.
Builds on
31 of 47 references.
References · 31
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2016Variational Inference: A Review for Statisticianscited by 1,466
- 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
- 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
- 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
- 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
- 2019A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networkscited by 213
- 2021Trustworthy AI: From Principles to Practicescited by 206
- 2015A Complete Recipe for Stochastic Gradient MCMCcited by 146
19 more
- 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
- 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 112
- 2024Variational Bayesian Last Layerscited by 67
- 2014Consistency and fluctuations for stochastic gradient Langevin dynamicscited by 65
- 2018The promises and pitfalls of Stochastic Gradient Langevin Dynamicscited by 56
- 2024BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Modelscited by 48
- 2016On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integratorscited by 35
- 2020Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noisecited by 33
- 2017Control Variates for Stochastic Gradient MCMCcited by 25
- 2013Austerity in MCMC Land: Cutting the Metropolis-Hastings Budgetcited by 24
- 2023Thermodynamic Computing System for AI Applicationscited by 16
- 2024Scalable Bayesian Learning with posteriorscited by 13
- 2019Thermodynamic Computingcited by 12
- 2025Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inferencecited by 11
- 2024Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgettingcited by 9
- 2025Solving the compute crisis with physics-based ASICscited by 8
- 2020Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradientscited by 2
- 2025Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimizationcited by 2
- 2025Mitigating Forgetting in Low Rank Adaptationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.