# Central limit theorem for the averaged Adam optimizer

arXiv: https://arxiv.org/abs/2606.21433 · HTML: https://mlsanity.com/paper/2606.21433/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 19 Jun 2026.

## Builds on (most cited first, 20 of 20)

- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [A Simple Convergence Proof of Adam and Adagrad](https://mlsanity.com/paper/2003.02395/) (2020)
- [The Road Less Scheduled](https://mlsanity.com/paper/2405.15682/) (2024)
- [Adam Can Converge Without Any Modification On Update Rules](https://mlsanity.com/paper/2208.09632/) (2022)
- [Adam with model exponential moving average is effective for nonconvex optimization](https://mlsanity.com/paper/2405.18199/) (2024)
- [Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory](https://mlsanity.com/paper/2310.20360/) (2023)
- [Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates](https://mlsanity.com/paper/2407.08100/) (2024)
- [Convergence rates for the Adam optimizer](https://mlsanity.com/paper/2407.21078/) (2024)
- [General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization](https://mlsanity.com/paper/2411.07061/) (2024)
- [On Constructing Confidence Region for Model Parameters in Stochastic Gradient Descent via Batch Means](https://mlsanity.com/paper/1911.01483/) (2019)
- [Lower error bounds for the stochastic gradient descent optimization algorithm: Sharp convergence rates for slowly and fast decaying learning rates](https://mlsanity.com/paper/1803.08600/) (2018)
- [ODE approximation for the Adam algorithm: General and overparametrized setting](https://mlsanity.com/paper/2511.04622/) (2025)
- [Blow up phenomena for gradient descent optimization methods in the training of artificial neural networks](https://mlsanity.com/paper/2211.15641/) (2022)
- [Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer](https://mlsanity.com/paper/2511.06675/) (2025)
- [SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures](https://mlsanity.com/paper/2505.09572/) (2025)
- [Averaged Adam accelerates stochastic optimization in the training of deep neural network approximations for partial differential equation and optimal control problems](https://mlsanity.com/paper/2501.06081/) (2025)
- [Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method](https://mlsanity.com/paper/2603.18899/) (2026)
- [A Short Survey of Averaging Techniques in Stochastic Gradient Methods](https://mlsanity.com/paper/2603.09634/) (2026)
- [PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning](https://mlsanity.com/paper/2505.22085/) (2025)

## Built on by (most cited first, 1 of 1)

- [Statistical Inference for Rank Allocation in Low-Rank Adaptation](https://mlsanity.com/paper/2607.20205/) (2026)

## Abstract (arXiv)

In this article, we analyse convergence of the averaged Adam optimizer to an attracting zero of the Adam vector field. We provide a central limit theorem that, in particular, quantifies exactly the speed of convergence. The order of convergence is n-1/2 in the number of steps of the algorithm which coincides with the order observed for classical stochastic approximation algorithms. The covariance in the central limit theorem is given in terms of properties of the Adam algorithm in the state of the attractor.