mlsanityData to 14 Oct 2026

Variational Structure at the Edge of Stability

arXiv 2608.21660 ↗First posted 21 Aug 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

When discrete-time optimizers operate at the edge of stability, they exhibit near-two-periodic behavior. These oscillatory dynamics are reminiscent of conservative systems, such as the dynamics generated by symplectic integrators. However, a precise formulation of the connection between discrete-time optimizers at the edge of stability and discrete mechanics remains underexplored. Recently, Litman introduced the "edge coupling": a functional on consecutive gradient descent iterates whose critical points encode the fixed points and two-point orbits of the gradient descent dynamics. Here we extend the edge coupling to heavy-ball and Nesterov momentum. We show that its critical points characterize the fixed points and two-point orbits, with its Hessian characterizing their stability. We also show that the edge coupling can be identified with the symmetric Verlet action, formalizing the connection between the edge of stability and discrete mechanics.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

16 of 25 references.

References · 16

  1. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
  2. 2022Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stabilitycited by 155
  3. 2020The Break-Even Point on Optimization Trajectories of Deep Neural Networkscited by 150
  4. 2022Understanding Gradient Descent on Edge of Stability in Deep Learningcited by 128
  5. 2022Adaptive Gradient Methods at the Edge of Stabilitycited by 101
  6. 2022Understanding the unstable convergence of gradient descentcited by 86
  7. 2024Understanding Optimization in Deep Learning with Central Flowscited by 64
  8. 2022Beyond the Edge of Stability via Two-step Gradient Updatescited by 38
  9. 2019Conformal Symplectic and Relativistic Optimizationcited by 29
  10. 2020Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectivescited by 27
  11. 2026Momentum Further Constrains Sharpness at the Edge of Stochastic Stabilitycited by 10
  12. 2026Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stabilitycited by 9
4 more
  1. 2026The Origin of Edge of Stabilitycited by 6
  2. 2023Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapultscited by 3
  3. 2026A Rod Flow Model for Adam at the Edge of Stabilitycited by 3
  4. 2026Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stabilitycited by 3

Built on by

Not cited yet.