# Gradient Descent in the Absence of Global Lipschitz Continuity of the Gradients

arXiv: https://arxiv.org/abs/2210.02418 · HTML: https://mlsanity.com/paper/2210.02418/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2022.

## Built on by (most cited first, 11 of 11)

- [Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion](https://mlsanity.com/paper/2302.03775/) (2023)
- [Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD](https://mlsanity.com/paper/2302.06570/) (2023)
- [Directional Smoothness and Gradient Methods: Convergence and Adaptivity](https://mlsanity.com/paper/2403.04081/) (2024)
- [Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learning](https://mlsanity.com/paper/2310.03461/) (2023)
- [Evaluating the diversity and utility of materials proposed by generative models](https://mlsanity.com/paper/2309.12323/) (2023)
- [Statistical Inference for Online Algorithms](https://mlsanity.com/paper/2505.17300/) (2025)
- [Glocal Smoothness: Line search and adaptive step sizes can help in theory too!](https://mlsanity.com/paper/2506.12648/) (2025)
- [First-ish Order Methods: Hessian-aware Scalings of Gradient Descent](https://mlsanity.com/paper/2502.03701/) (2025)
- [Stabilizing Multimodal Autoencoders: A Theoretical and Empirical Analysis of Fusion Strategies](https://mlsanity.com/paper/2512.20749/) (2025)
- [Recent Advances in Non-convex Smoothness Conditions and Applicability to Deep Linear Neural Networks](https://mlsanity.com/paper/2409.13672/) (2024)
- [Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization](https://mlsanity.com/paper/2604.12768/) (2026)

## Abstract (arXiv)

Gradient descent (GD) is a collection of continuous optimization methods that have achieved immeasurable success in practice. Owing to data science applications, GD with diminishing step sizes has become a prominent variant. While this variant of GD has been well-studied in the literature for objectives with globally Lipschitz continuous gradients or by requiring bounded iterates, objectives from data science problems do not satisfy such assumptions. Thus, in this work, we provide a novel global convergence analysis of GD with diminishing step sizes for differentiable nonconvex functions whose gradients are only locally Lipschitz continuous. Through our analysis, we generalize what is known about gradient descent with diminishing step sizes including interesting topological facts; and we elucidate the varied behaviors that can occur in the previously overlooked divergence regime. Thus, we provide the most general global convergence analysis of GD with diminishing step sizes under realistic conditions for data science problems.