# Beyond Convexity -- Contraction and Global Convergence of Gradient Descent

arXiv: https://arxiv.org/abs/1806.06655 · HTML: https://mlsanity.com/paper/1806.06655/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2018.

## Built on by (most cited first, 17 of 17)

- [Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overview](https://mlsanity.com/paper/2110.00675/) (2021)
- [Adaptive-Control-Oriented Meta-Learning for Nonlinear Systems](https://mlsanity.com/paper/2103.04490/) (2021)
- [On the Sample Complexity of Stability Constrained Imitation Learning](https://mlsanity.com/paper/2102.09161/) (2021)
- [Control-oriented meta-learning](https://mlsanity.com/paper/2204.06716/) (2022)
- [Learning Neural Contracting Dynamics: Extended Linearization and Global Guarantees](https://mlsanity.com/paper/2402.08090/) (2024)
- [Generalization in Supervised Learning Through Riemannian Contraction](https://mlsanity.com/paper/2201.06656/) (2022)
- [RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks](https://mlsanity.com/paper/2106.08928/) (2021)
- [A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows](https://mlsanity.com/paper/2105.08832/) (2021)
- [Learning Dynamical Systems Encoding Non-Linearity within Space Curvature](https://mlsanity.com/paper/2403.11948/) (2024)
- [Is All Learning (Natural) Gradient Descent?](https://mlsanity.com/paper/2409.16422/) (2024)
- [Characterizing control between interacting subsystems with deep Jacobian estimation](https://mlsanity.com/paper/2507.01946/) (2025)
- [Convergence Properties of Natural Gradient Descent for Minimizing KL Divergence](https://mlsanity.com/paper/2504.19259/) (2025)
- [Similarity Matching Networks: Hebbian Learning and Convergence Over Multiple Time Scales](https://mlsanity.com/paper/2506.06134/) (2025)
- [Thermodynamic structure of the Sinkhorn flow](https://mlsanity.com/paper/2510.12639/) (2025)
- [A Systems-Theoretic View on the Convergence of Algorithms under Disturbances](https://mlsanity.com/paper/2512.17598/) (2025)
- [Riemannian Optimization in Modular Systems](https://mlsanity.com/paper/2603.03610/) (2026)
- [A Closed-Form Upper Bound for Admissible Learning-Rate Steps in Belief-Space Dynamics](https://mlsanity.com/paper/2605.06741/) (2026)

## Abstract (arXiv)

This paper considers the analysis of continuous time gradient-based optimization algorithms through the lens of nonlinear contraction theory. It demonstrates that in the case of a time-invariant objective, most elementary results on gradient descent based on convexity can be replaced by much more general results based on contraction. In particular, gradient descent converges to a unique equilibrium if its dynamics are contracting in any metric, with convexity of the cost corresponding to the special case of contraction in the identity metric. More broadly, contraction analysis provides new insights for the case of geodesically-convex optimization, wherein non-convex problems in Euclidean space can be transformed to convex ones posed over a Riemannian manifold. In this case, natural gradient descent converges to a unique equilibrium if it is contracting in any metric, with geodesic convexity of the cost corresponding to contraction in the natural metric. New results using semi-contraction provide additional insights into the topology of the set of optimizers in the case when multiple optima exist. Furthermore, they show how semi-contraction may be combined with specific additional information to reach broad conclusions about a dynamical system. The contraction perspective also easily extends to time-varying optimization settings and allows one to recursively build large optimization structures out of simpler elements. Extensions to natural primal-dual optimization and game-theoretic contexts further illustrate the potential reach of these new perspectives.