# Adam at the Edge of Stability: Adaptive Feedback, Provable Oscillation, and Gradient Reversal

arXiv: https://arxiv.org/abs/2608.20638 · HTML: https://mlsanity.com/paper/2608.20638/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 20 of 20)

- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [The large learning rate phase of deep learning: the catapult mechanism](https://mlsanity.com/paper/2003.02218/) (2020)
- [Adam Can Converge Without Any Modification On Update Rules](https://mlsanity.com/paper/2208.09632/) (2022)
- [Adaptive Gradient Methods at the Edge of Stability](https://mlsanity.com/paper/2207.14484/) (2022)
- [On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length](https://mlsanity.com/paper/1807.05031/) (2018)
- [Understanding the unstable convergence of gradient descent](https://mlsanity.com/paper/2204.01050/) (2022)
- [Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability](https://mlsanity.com/paper/2207.12678/) (2022)
- [Learning threshold neurons via the "edge of stability"](https://mlsanity.com/paper/2212.07469/) (2022)
- [Convergence and Dynamical Behavior of the ADAM Algorithm for Non-Convex Stochastic Optimization](https://mlsanity.com/paper/1810.02263/) (2018)
- [Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD](https://mlsanity.com/paper/2412.20553/) (2024)
- [Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory](https://mlsanity.com/paper/2307.04204/) (2023)
- [Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos](https://mlsanity.com/paper/2311.02076/) (2023)
- [A Defense of the Quadratic Model](https://mlsanity.com/paper/2607.21716/) (2026)
- [Adaptive Preconditioners Trigger Loss Spikes in Adam](https://mlsanity.com/paper/2506.04805/) (2025)
- [Local Convergence of Adaptive Gradient Descent Optimizers](https://mlsanity.com/paper/2102.09804/) (2021)
- [Adam Reduces a Unique Form of Sharpness: Theoretical Insights Near the Minimizer Manifold](https://mlsanity.com/paper/2511.02773/) (2025)
- [Asymptotic stability properties and a priori bounds for Adam and other gradient descent optimization methods](https://mlsanity.com/paper/2509.10476/) (2025)
- [Towards Understanding Adam Convergence on Highly Degenerate Polynomials](https://mlsanity.com/paper/2603.09581/) (2026)
- [A Rod Flow Model for Adam at the Edge of Stability](https://mlsanity.com/paper/2605.06821/) (2026)
- [State-Dependent Lyapunov Analysis of Rank-1 Matrix Factorization](https://mlsanity.com/paper/2604.26993/) (2026)

## Built on by (most cited first, 1 of 1)

- [Beyond Quadratic Loss: The Stability Phase Diagram of Adam](https://mlsanity.com/paper/2609.18314/) (2026)

## Abstract (arXiv)

The edge-of-stability (EoS) phenomenon of full-batch Adam has been widely observed, yet its underlying dynamical mechanism remains poorly understood. In this paper, we identify Adam's second-moment adaptation as a negative-feedback mechanism that drives the dynamics toward the stability boundary. We characterize this mechanism through the *active curvature*, namely, the preconditioned curvature along the preconditioned gradient direction, and establish rigorous characterizations in progressively richer settings: rank-one quadratics with momentum, diagonal quadratics, on which the active curvature separates from the sharpness, and general objectives. Importantly, the mechanism predicts *gradient reversal* of full-batch Adam near the edge: consecutive gradients repeatedly point in nearly opposite directions, as we observe across fully connected networks, ResNets, ViTs, LSTMs, GPT-2 medium, and Adam-family optimizers. Consistent with this picture, averaging iterates suppresses these fast oscillations and produces smoother and lower loss curves. Together, these results provide an important first step towards fully understanding the dynamical behavior of Adam's EoS through active curvature and gradient reversal.