mlsanityData to 13 Oct 2026

Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example

arXiv 2308.00720 ↗First posted 1 Aug 2023cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

A very simple unidimensional function with Lipschitz continuous gradient is constructed such that the ADAM algorithm with constant stepsize, started from the origin, diverges when applied to minimize this function in the absence of noise on the gradient. Divergence occurs irrespective of the choice of the method parameters.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 4 of its 5 references that are in the map. First posted on arXiv 1 Aug 2023.

Builds on

4 of 5 references.

References · 4

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2019On the Convergence of Adam and Beyondcited by 837
  3. 2020A Simple Convergence Proof of Adam and Adagradcited by 169
  4. 2022Adam Can Converge Without Any Modification On Update Rulescited by 114

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026A unified convergence theory for adaptive first-order methods in the nonconvex case, including AdaNorm, full and diagonal AdaGrad and Muoncited by 2
  2. 2026On the Convergence of Adam, Revisitedcited by 1