mlsanityData to 7 Oct 2026

Convergence of continuous-time stochastic gradient descent with applications to deep neural networks

arXiv 2409.07401 ↗First posted 11 Sep 2024cs.LG math.OC stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

We study a continuous-time approximation of the stochastic gradient descent process for minimizing the population expected loss in learning problems. The main results establish general sufficient conditions for the convergence, extending the results of Chatterjee (2022) established for (nonstochastic) gradient descent. We show how the main result can be applied to the case of overparametrized neural network training.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 11 Sep 2024.

Builds on

12 of 18 references.

References · 12

  1. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  2. 2017How to Escape Saddle Points Efficientlycited by 284
  3. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
  4. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  5. 2019Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizerscited by 61
  6. 2022Convergence of gradient descent for deep neural networkscited by 24
  7. 2021Convergence of gradient descent for learning linear neural networkscited by 22
  8. 2023Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descentcited by 8
  9. 2022From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descentcited by 5
  10. 2023Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networkscited by 3
  11. 2024Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descentcited by 3
  12. 2021Higher Order Generalization Error for First Order Discretization of Langevin Diffusioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equationcited by 2
  2. 2026From Adam to Adam-Like Lagrangians: Second-Order Nonlocal Dynamicscited by 1
  3. 2025Convergence of Stochastic Gradient Langevin Dynamics in the Lazy Training Regimecited by 0
  4. 2026Generalization Bounds of Stochastic Gradient Descent in Homogeneous Neural Networkscited by 0