mlsanityData to 14 Oct 2026

Ito Diffusion Approximation of Universal Ito Chains for Sampling, Optimization and Boosting

arXiv 2310.06081 ↗First posted 9 Oct 2023math.OC cs.LG math.PRcited by 0

Not on the map. Why?

Abstract (arXiv)

In this work, we consider rather general and broad class of Markov chains, Ito chains, that look like Euler-Maryama discretization of some Stochastic Differential Equation. The chain we study is a unified framework for theoretical analysis. It comes with almost arbitrary isotropic and state-dependent noise instead of normal and state-independent one as in most related papers. Moreover, in our chain the drift and diffusion coefficient can be inexact in order to cover wide range of applications as Stochastic Gradient Langevin Dynamics, sampling, Stochastic Gradient Descent or Stochastic Gradient Boosting. We prove the bound in W2-distance between the laws of our Ito chain and corresponding differential equation. These results improve or cover most of the known estimates. And for some particular cases, our analysis is the first.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 61 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

25 of 61 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
  5. 2019A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networkscited by 214
  6. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 208
  7. 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 198
  8. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 146
  9. 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
  10. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 127
  11. 2019Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like processcited by 123
  12. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
13 more
  1. 2018Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundationscited by 114
  2. 2018Sharp convergence rates for Langevin dynamics in the nonconvex settingcited by 68
  3. 2018Sampling Can Be Faster Than Optimizationcited by 68
  4. 2018Global Non-convex Optimization with Discretized Diffusionscited by 44
  5. 2019Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyondcited by 33
  6. 2017Efficient Regret Minimization in Non-Convex Gamescited by 28
  7. 2019Langevin Monte Carlo without smoothnesscited by 28
  8. 2018Continuous-time Models for Stochastic Optimization Algorithmscited by 17
  9. 2019Uniform-in-Time Weak Error Analysis for Stochastic Gradient Descent Algorithms via Diffusion Approximationcited by 15
  10. 2020SGLB: Stochastic Gradient Langevin Boostingcited by 12
  11. 2020Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Driftscited by 12
  12. 2020Dimension-free convergence rates for gradient Langevin dynamics in RKHScited by 6
  13. 2019Adaptively Preconditioned Stochastic Gradient Langevin Dynamicscited by 5

Built on by

Not cited yet.