mlsanityData to 7 Oct 2026

An improvement of the convergence proof of the ADAM-Optimizer

arXiv 1804.10587 ↗First posted 27 Apr 2018cs.LG cs.AI stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

A common way to train neural networks is the Backpropagation. This algorithm includes a gradient descent method, which needs an adaptive step size. In the area of neural networks, the ADAM-Optimizer is one of the most popular adaptive step size methods. It was invented in citeKingma.2015 by Kingma and Ba. The 5865 citations in only three years shows additionally the importance of the given paper. We discovered that the given convergence proof of the optimizer contains some mistakes, so that the proof will be wrong. In this paper we give an improvement to the convergence proof of the ADAM-Optimizer.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 2; 2023: 2; 2024: 3; 2025: 3; 2026: 4.

19 papers.

Sort:

  1. 2021Local Convergence of Adaptive Gradient Descent Optimizerscited by 4
  2. 2022Predicting Drug-Drug Interactions using Deep Generative Models on Graphscited by 4
  3. 2023The Forward-Forward Algorithm as a feature extractor for skin lesion classification: A preliminary studycited by 3
  4. 2025Stochastic Approximation with Block Coordinate Optimal Stepsizescited by 3
  5. 2021p2pGNN: A Decentralized Graph Neural Network for Node Classification in Peer-to-Peer Networkscited by 2
  6. 2026Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization methodcited by 2
  7. 2021A decreasing scaling transition scheme from Adam to SGDcited by 1
  8. 2021Extending AdamW by Leveraging Its Second Moment and Magnitudecited by 1
  9. 2025WMINet: A Wheel-Mounted Inertial Learning Approach For Mobile-Robot Positioningcited by 1
  10. 2025Gradient Descent with Provably Tuned Learning-rate Schedulescited by 1
  11. 2026DP-MacAdam: Differentially Private Mechanism with Adaptive Clipping and Adaptive Momentumcited by 1
  12. 2021On a Conjecture Regarding the Adam Optimizercited by 0
7 more
  1. 2022Simulating Using Deep Learning The World Trade Forecasting of Export-Import Exchange Rate Convergence Factor During COVID-19cited by 0
  2. 2023Based on What We Can Control Artificial Neural Networkscited by 0
  3. 2024Advancing Generative Model Evaluation: A Novel Algorithm for Realistic Image Synthesis and Comparison in OCR Systemcited by 0
  4. 2024GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithmscited by 0
  5. 2024Prediction of Cardiovascular Risk Factors from Retinal Fundus Images using CNNscited by 0
  6. 2026Using Laplace Transform To Optimize the Hallucination of Generation Modelscited by 0
  7. 2026Single-Beat Cuffless Blood Pressure Estimation Using Ear-PPG and ECG with a Lightweight Hybrid Learning Frameworkcited by 0