Deterministic Denominator Design for Localized Tamed Stochastic Gradient Langevin Dynamics
Not on the map. Why?
Abstract (arXiv)
If the denominator in a tamed stochastic gradient Langevin update uses the current stochastic-gradient draw, the conditional mean can be biased even when the stochastic-gradient oracle is unbiased. A state-dependent denominator fixed before that draw removes this coupling. We build practical deterministic denominators from a short pilot run. A log-scale proxy is fitted to the growth score G_star(x)=|b(x)|/(1+|x|), and empirical pilot quantiles set the activation thresholds of a local proxy-quantile envelope. The reported proxy-quantile experiments use this local denominator. Separately, we describe a final denominator with a norm-polynomial tail-floor correction that can be used when one wants to certify the global effective-linearity input required by the companion deterministic-envelope Lyapunov theory. We show how proxy and threshold errors enter denominator errors and the resulting stationary observable errors. In the reported experiments, the local proxy-quantile denominator improves over random-denominator tamed SGLD at comparable production cost. It also gives observable behavior close to the G_star-envelope benchmark, without the full-gradient growth-score evaluations required by that benchmark.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 29 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
14 of 29 references.
References · 14
- 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
- 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
- 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 208
- 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 197
- 2015A Complete Recipe for Stochastic Gradient MCMCcited by 146
- 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
- 2023Revisiting Gradient Clipping: Stochastic bias and tight convergence guaranteescited by 83
- 2018The promises and pitfalls of Stochastic Gradient Langevin Dynamicscited by 56
- 2020Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Samplingcited by 32
- 2017The Tamed Unadjusted Langevin Algorithmcited by 31
- 2020Taming neural networks with TUSLA: Non-convex learning via adaptive stochastic gradient Langevin algorithmscited by 21
- 2015Perturbation theory for Markov chains via Wasserstein distancecited by 18
Built on by
Citing papers per year: 2026: 1.
1 paper.