# Precise Convergence Speed of Clipped SGD

arXiv: https://arxiv.org/abs/2609.29458 · HTML: https://mlsanity.com/paper/2609.29458/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 6 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 5 of 5)

- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Why gradient clipping accelerates training: A theoretical justification for adaptivity](https://mlsanity.com/paper/1905.11881/) (2019)
- [Improved Analysis of Clipping Algorithms for Non-convex Optimization](https://mlsanity.com/paper/2010.02519/) (2020)
- [Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees](https://mlsanity.com/paper/2305.01588/) (2023)
- [Avoiding Bias in Clipped SGD for Overparameterized Models under Generalized Smoothness](https://mlsanity.com/paper/2605.14800/) (2026)

## Abstract (arXiv)

We present a tightened convergence analysis of clipped gradient descent on (L0, L1)-smooth functions, with quantitative constants. Building on the ideas of Koloskova et al (2023), we refactor several case disjunctions to reveal the central role of a control of the bias derived from fundamental properties of ℓ2-projection, simplifying proofs. We also extend the domain of validity from η ≤ 1 / (9 β) to η < 1 /β where β = L0 + c L1 for clipping constant c, which matches the more traditional analysis of smooth functions. We strengthen the convergence criterion from left( mint < T mathbbE[lVert nabla f(xt) rVert2] right) to left( frac1T sumt < T mathbbE[lVert nabla f(xt) rVert2] right) with matching speed, and lower the final achievable loss from O(min(σ2/c, σ)) to the more precise 6 min(σ2 /c, 3 σ).