# Toward a Unified Theory of Gradient Descent under Generalized Smoothness

arXiv: https://arxiv.org/abs/2412.11773 · HTML: https://mlsanity.com/paper/2412.11773/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2024.

## Built on by (most cited first, 5 of 5)

- [On the Interaction of Batch Noise, Adaptivity, and Compression, under (L0,L1)-Smoothness: An SDE Approach](https://mlsanity.com/paper/2506.00181/) (2025)
- [Theoretical Analysis on how Learning Rate Warmup Accelerates Convergence](https://mlsanity.com/paper/2509.07972/) (2025)
- [Gradient-Normalized Smoothness for Optimization with Approximate Hessians](https://mlsanity.com/paper/2506.13710/) (2025)
- [Near-Optimal Convergence of Accelerated Gradient Methods under Generalized and (L0, L1)-Smoothness](https://mlsanity.com/paper/2508.06884/) (2025)
- [Decentralized Stochastic Nonconvex Optimization under the (L0,L1)-Smoothness](https://mlsanity.com/paper/2509.08726/) (2025)

## Abstract (arXiv)

We study the classical optimization problem minx ∈ ℝd f(x) and analyze the gradient descent (GD) method in both nonconvex and convex settings. It is well-known that, under the L-smoothness assumption (|nabla2 f(x)| ≤ L), the optimal point minimizing the quadratic upper bound f(xk) + langlenabla f(xk), xk+1 - xkrangle + fracL2 |xk+1 - xk|2 is xk+1 = xk - γk nabla f(xk) with step size γk = frac1L. Surprisingly, a similar result can be derived under the ℓ-generalized smoothness assumption (|nabla2 f(x)| ≤ ℓ(|nabla f(x)|)). In this case, we derive the step size γk = int01 fracd vℓ(|nabla f(xk)| + |nabla f(xk)| v). Using this step size rule, we improve upon existing theoretical convergence rates and obtain new results in several previously unexplored setups.