# Convergence of SGD with momentum in the nonconvex case: A time window-based analysis

arXiv: https://arxiv.org/abs/2405.16954 · HTML: https://mlsanity.com/paper/2405.16954/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2024.

## Built on by (most cited first, 4 of 4)

- [A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods](https://mlsanity.com/paper/2406.02273/) (2024)
- [Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks](https://mlsanity.com/paper/2607.04233/) (2026)
- [Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization](https://mlsanity.com/paper/2607.08104/) (2026)
- [Stochastic Gradient Descent Revisited](https://mlsanity.com/paper/2412.06070/) (2024)

## Abstract (arXiv)

The stochastic gradient descent method with momentum (SGDM) is a common approach for solving large-scale and stochastic optimization problems. Despite its popularity, the convergence behavior of SGDM remains less understood in nonconvex scenarios. This is primarily due to the absence of a sufficient descent property and challenges in simultaneously controlling the momentum and stochastic errors in an almost sure sense. To address these challenges, we investigate the behavior of SGDM over specific time windows, rather than examining the descent of consecutive iterates as in traditional studies. This time window-based approach simplifies the convergence analysis and enables us to establish the iterate convergence result for SGDM under the Lojasiewicz property. We further provide local convergence rates which depend on the underlying Lojasiewicz exponent and the utilized step size schemes.