# Non-Euclidean Gradient Descent Has Many Edges of Stability

arXiv: https://arxiv.org/abs/2610.15223 · HTML: https://mlsanity.com/paper/2610.15223/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 13 references that are in the map.
- First posted on arXiv 10 Oct 2026.

## Builds on (most cited first, 9 of 9)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability](https://mlsanity.com/paper/2103.00065/) (2021)
- [Old Optimizer, New Norm: An Anthology](https://mlsanity.com/paper/2409.20325/) (2024)
- [Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability](https://mlsanity.com/paper/2209.15594/) (2022)
- [Adaptive Gradient Methods at the Edge of Stability](https://mlsanity.com/paper/2207.14484/) (2022)
- [Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory](https://mlsanity.com/paper/2307.04204/) (2023)
- [Directional Smoothness and Gradient Methods: Convergence and Adaptivity](https://mlsanity.com/paper/2403.04081/) (2024)
- [Muon is Not That Special: Random or Inverted Spectra Work Just as Well](https://mlsanity.com/paper/2605.11181/) (2026)
- [Non-Euclidean Gradient Descent Operates at the Edge of Stability](https://mlsanity.com/paper/2603.05002/) (2026)

## Abstract (arXiv)

Recent observations that deep learning models train at the edge of stability have renewed interest in understanding stability properties of classical optimization algorithms. For quadratic objectives, standard gradient descent has a sharp stability threshold: it converges from every initialization when η<2/L, diverges from almost every initialization when η>2/L, and diverges from every nonzero initialization when η>2/μ. For non-Euclidean gradient descent, the corresponding known guarantees are weaker: convergence holds for every initialization when η<2/S, where S is the generalized sharpness, while every nonzero initialization diverges when η>2/μ. The behavior in the intermediate regime 2/S<η<2/μ is not fully characterized. We show that Hilbertian norms are the only ones for which 2/S is an almost-everywhere sharp threshold for every positive-definite quadratic objective. Using linear conjugacy between non-Euclidean and Euclidean gradient descent, we show that for general norms the intermediate regime can contain arbitrarily many edges of stability associated with distinct attracting directions. We identify a forward-invariant region of exponentially divergent directions beyond the generalized stability threshold. When the directions of minimum curvature are finite and nondegenerate, we prove that, sufficiently close to 2/μ from below, only their invariant directions remain convergent. We give smooth two-dimensional constructions with arbitrarily many distinct thresholds, arbitrarily large relative threshold separation and show these properties to be stable.