# The convergence of the Stochastic Gradient Descent (SGD) : a self-contained proof

arXiv: https://arxiv.org/abs/2103.14350 · HTML: https://mlsanity.com/paper/2103.14350/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 5 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 4 of 4)

- [Stochastic Runge-Kutta methods and adaptive SGD-G2 stochastic gradient descent](https://mlsanity.com/paper/2002.09304/) (2020)
- [Radon Sobolev Variational Auto-Encoders](https://mlsanity.com/paper/1911.13135/) (2019)
- [Stochastic learning control of inhomogeneous quantum ensembles](https://mlsanity.com/paper/1906.02991/) (2019)
- [Convergence dynamics of Generative Adversarial Networks: the dual metric flows](https://mlsanity.com/paper/2012.10410/) (2020)

## Built on by (most cited first, 9 of 9)

- [A Bregman Learning Framework for Sparse Neural Networks](https://mlsanity.com/paper/2105.04319/) (2021)
- [Optimal time sampling in physics-informed neural networks](https://mlsanity.com/paper/2404.18780/) (2024)
- [Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Bandit](https://mlsanity.com/paper/2402.06388/) (2024)
- [Analysis of Kinetic Models for Label Switching and Stochastic Gradient Descent](https://mlsanity.com/paper/2207.00389/) (2022)
- [Prediction intervals for neural network models using weighted asymmetric loss functions](https://mlsanity.com/paper/2210.04318/) (2022)
- [A Scale-Independent Multi-Objective Reinforcement Learning with Convergence Analysis](https://mlsanity.com/paper/2302.04179/) (2023)
- [Learning Continually on a Sequence of Graphs -- The Dynamical System Way](https://mlsanity.com/paper/2305.12030/) (2023)
- [Onflow: a model free, online portfolio allocation algorithm robust to transaction fees](https://mlsanity.com/paper/2312.05169/) (2023)
- [Probabilistic and nonlinear compressive sensing](https://mlsanity.com/paper/2509.15060/) (2025)

## Abstract (arXiv)

We give here a proof of the convergence of the Stochastic Gradient Descent (SGD) in a self-contained manner.