# Gradient Descent Converges to Minimizers

arXiv: https://arxiv.org/abs/1602.04915 · HTML: https://mlsanity.com/paper/1602.04915/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2016.

## Built on by (most cited first, 25 of 25)

- [Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok](https://mlsanity.com/paper/2306.13253/) (2023)
- [Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature](https://mlsanity.com/paper/2102.04168/) (2021)
- [Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time](https://mlsanity.com/paper/2306.16361/) (2023)
- [Visualizing high-dimensional loss landscapes with Hessian directions](https://mlsanity.com/paper/2208.13219/) (2022)
- [Stochastic noise can be helpful for variational quantum algorithms](https://mlsanity.com/paper/2210.06723/) (2022)
- [Optimal Rate Adaption in Federated Learning with Compressed Communications](https://mlsanity.com/paper/2112.06694/) (2021)
- [Offline Policy Evaluation and Optimization under Confounding](https://mlsanity.com/paper/2211.16583/) (2022)
- [Special Properties of Gradient Descent with Large Learning Rates](https://mlsanity.com/paper/2205.15142/) (2022)
- [Why Do Local Methods Solve Nonconvex Problems?](https://mlsanity.com/paper/2103.13462/) (2021)
- [Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding](https://mlsanity.com/paper/2402.05626/) (2024)
- [Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance](https://mlsanity.com/paper/2604.16620/) (2026)
- [Escaping Saddle Points in Distributed Newton's Method with Communication Efficiency and Byzantine Resilience](https://mlsanity.com/paper/2103.09424/) (2021)
- [Distributed stochastic optimization with large delays](https://mlsanity.com/paper/2107.02919/) (2021)
- [The Landscape of Multi-Layer Linear Neural Network From the Perspective of Algebraic Geometry](https://mlsanity.com/paper/2102.04338/) (2021)
- [CoShNet: A Hybrid Complex Valued Neural Network using Shearlets](https://mlsanity.com/paper/2208.06882/) (2022)
- [Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning](https://mlsanity.com/paper/2408.13767/) (2024)
- [Implicit Bias in Matrix Factorization and its Explicit Realization in a New Architecture](https://mlsanity.com/paper/2501.16322/) (2025)
- [Boundary Conditions for Linear Exit Time Gradient Trajectories Around Saddle Points: Analysis and Algorithm](https://mlsanity.com/paper/2101.02625/) (2021)
- [Asymptotic Escape of Spurious Critical Points on the Low-rank Matrix Manifold](https://mlsanity.com/paper/2107.09207/) (2021)
- [Accelerated Gradient Flow: Risk, Stability, and Implicit Regularization](https://mlsanity.com/paper/2201.08311/) (2022)
- [Towards guarantees for parameter isolation in continual learning](https://mlsanity.com/paper/2310.01165/) (2023)
- [Harmonic Control Lyapunov Barrier Functions for Constrained Optimal Control with Reach-Avoid Specifications](https://mlsanity.com/paper/2310.02869/) (2023)
- [Local vs Global continual learning](https://mlsanity.com/paper/2407.16611/) (2024)
- [k-SVD with Gradient Descent](https://mlsanity.com/paper/2502.00320/) (2025)
- [Riemannian Optimization for Multi-Player Quantum Games on Product Unitary Manifolds](https://mlsanity.com/paper/2609.07397/) (2026)

## Abstract (arXiv)

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.