mlsanityData to 14 Oct 2026

Central limit theorem for the averaged Adam optimizer

arXiv 2606.21433 ↗First posted 19 Jun 2026math.PR cs.LG math.STcited by 1

Not on the map. Why?

Abstract (arXiv)

In this article, we analyse convergence of the averaged Adam optimizer to an attracting zero of the Adam vector field. We provide a central limit theorem that, in particular, quantifies exactly the speed of convergence. The order of convergence is n-1/2 in the number of steps of the algorithm which coincides with the order observed for classical stochastic approximation algorithms. The covariance in the central limit theorem is given in terms of properties of the Adam algorithm in the state of the attractor.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 19 Jun 2026.

Builds on

20 of 35 references.

References · 20

  1. 2016An overview of gradient descent optimization algorithmscited by 1,033
  2. 2019On the Convergence of Adam and Beyondcited by 841
  3. 2020A Simple Convergence Proof of Adam and Adagradcited by 171
  4. 2024The Road Less Scheduledcited by 163
  5. 2022Adam Can Converge Without Any Modification On Update Rulescited by 114
  6. 2024Adam with model exponential moving average is effective for nonconvex optimizationcited by 23
  7. 2023Mathematical Introduction to Deep Learning: Methods, Implementations, and Theorycited by 20
  8. 2024Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning ratescited by 15
  9. 2024Convergence rates for the Adam optimizercited by 15
  10. 2024General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimizationcited by 15
  11. 2019On Constructing Confidence Region for Model Parameters in Stochastic Gradient Descent via Batch Meanscited by 13
  12. 2018Lower error bounds for the stochastic gradient descent optimization algorithm: Sharp convergence rates for slowly and fast decaying learning ratescited by 11
8 more
  1. 2025ODE approximation for the Adam algorithm: General and overparametrized settingcited by 10
  2. 2022Blow up phenomena for gradient descent optimization methods in the training of artificial neural networkscited by 9
  3. 2025Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizercited by 6
  4. 2025SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structurescited by 5
  5. 2025Averaged Adam accelerates stochastic optimization in the training of deep neural network approximations for partial differential equation and optimal control problemscited by 3
  6. 2026Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization methodcited by 3
  7. 2026A Short Survey of Averaging Techniques in Stochastic Gradient Methodscited by 2
  8. 2025PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Statistical Inference for Rank Allocation in Low-Rank Adaptationcited by 0