# Stochastic Gradient Descent on Nonconvex Functions with General Noise Models

arXiv: https://arxiv.org/abs/2104.00423 · HTML: https://mlsanity.com/paper/2104.00423/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2021.

## Built on by (most cited first, 7 of 7)

- [Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity](https://mlsanity.com/paper/2107.00052/) (2021)
- [Global Convergence and Stability of Stochastic Gradient Descent](https://mlsanity.com/paper/2110.01663/) (2021)
- [Online Decision-Focused Learning](https://mlsanity.com/paper/2505.13564/) (2025)
- [Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignment](https://mlsanity.com/paper/2406.11039/) (2024)
- [On Uniform Boundedness Properties of SGD and its Momentum Variants](https://mlsanity.com/paper/2201.10245/) (2022)
- [Online (Non-)Convex Learning via Tempered Optimism](https://mlsanity.com/paper/2301.07530/) (2023)
- [Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H"older Smoothness](https://mlsanity.com/paper/2609.12785/) (2026)

## Abstract (arXiv)

Stochastic Gradient Descent (SGD) is a widely deployed optimization procedure throughout data-driven and simulation-driven disciplines, which has drawn a substantial interest in understanding its global behavior across a broad class of nonconvex problems and noise models. Recent analyses of SGD have made noteworthy progress in this direction, and these analyses have innovated important and insightful new strategies for understanding SGD. However, these analyses often have imposed certain restrictions (e.g., convexity, global Lipschitz continuity, uniform Holder continuity, expected smoothness, etc.) that leave room for innovation. In this work, we address this gap by proving that, for a rather general class of nonconvex functions and noise models, SGD's iterates either diverge to infinity or converge to a stationary point with probability one. By further restricting to globally Holder continuous functions and the expected smoothness noise model, we prove that -- regardless of whether the iterates diverge or remain finite -- the norm of the gradient function evaluated at SGD's iterates converges to zero with probability one and in expectation. As a result of our work, we broaden the scope of nonconvex problems and noise models to which SGD can be applied with rigorous guarantees of its global behavior.