# An improvement of the convergence proof of the ADAM-Optimizer

arXiv: https://arxiv.org/abs/1804.10587 · HTML: https://mlsanity.com/paper/1804.10587/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2018.

## Built on by (most cited first, 19 of 19)

- [Local Convergence of Adaptive Gradient Descent Optimizers](https://mlsanity.com/paper/2102.09804/) (2021)
- [Predicting Drug-Drug Interactions using Deep Generative Models on Graphs](https://mlsanity.com/paper/2209.09941/) (2022)
- [The Forward-Forward Algorithm as a feature extractor for skin lesion classification: A preliminary study](https://mlsanity.com/paper/2307.00617/) (2023)
- [Stochastic Approximation with Block Coordinate Optimal Stepsizes](https://mlsanity.com/paper/2507.08963/) (2025)
- [Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method](https://mlsanity.com/paper/2603.18899/) (2026)
- [p2pGNN: A Decentralized Graph Neural Network for Node Classification in Peer-to-Peer Networks](https://mlsanity.com/paper/2111.14837/) (2021)
- [A decreasing scaling transition scheme from Adam to SGD](https://mlsanity.com/paper/2106.06749/) (2021)
- [Extending AdamW by Leveraging Its Second Moment and Magnitude](https://mlsanity.com/paper/2112.06125/) (2021)
- [WMINet: A Wheel-Mounted Inertial Learning Approach For Mobile-Robot Positioning](https://mlsanity.com/paper/2503.13568/) (2025)
- [Gradient Descent with Provably Tuned Learning-rate Schedules](https://mlsanity.com/paper/2512.05084/) (2025)
- [DP-MacAdam: Differentially Private Mechanism with Adaptive Clipping and Adaptive Momentum](https://mlsanity.com/paper/2606.05435/) (2026)
- [On a Conjecture Regarding the Adam Optimizer](https://mlsanity.com/paper/2111.08162/) (2021)
- [Simulating Using Deep Learning The World Trade Forecasting of Export-Import Exchange Rate Convergence Factor During COVID-19](https://mlsanity.com/paper/2201.12291/) (2022)
- [Based on What We Can Control Artificial Neural Networks](https://mlsanity.com/paper/2310.05692/) (2023)
- [Advancing Generative Model Evaluation: A Novel Algorithm for Realistic Image Synthesis and Comparison in OCR System](https://mlsanity.com/paper/2402.17204/) (2024)
- [GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithms](https://mlsanity.com/paper/2405.16255/) (2024)
- [Prediction of Cardiovascular Risk Factors from Retinal Fundus Images using CNNs](https://mlsanity.com/paper/2410.11535/) (2024)
- [Using Laplace Transform To Optimize the Hallucination of Generation Models](https://mlsanity.com/paper/2603.18022/) (2026)
- [Single-Beat Cuffless Blood Pressure Estimation Using Ear-PPG and ECG with a Lightweight Hybrid Learning Framework](https://mlsanity.com/paper/2607.27076/) (2026)

## Abstract (arXiv)

A common way to train neural networks is the Backpropagation. This algorithm includes a gradient descent method, which needs an adaptive step size. In the area of neural networks, the ADAM-Optimizer is one of the most popular adaptive step size methods. It was invented in citeKingma.2015 by Kingma and Ba. The 5865 citations in only three years shows additionally the importance of the given paper. We discovered that the given convergence proof of the optimizer contains some mistakes, so that the proof will be wrong. In this paper we give an improvement to the convergence proof of the ADAM-Optimizer.