# Convergence of Stochastic Proximal Gradient Algorithm

arXiv: https://arxiv.org/abs/1403.5074 · HTML: https://mlsanity.com/paper/1403.5074/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2014.

## Built on by (most cited first, 17 of 17)

- [Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD](https://mlsanity.com/paper/2302.06570/) (2023)
- [Implicit Diffusion: Efficient Optimization through Stochastic Sampling](https://mlsanity.com/paper/2402.05468/) (2024)
- [A Bregman Learning Framework for Sparse Neural Networks](https://mlsanity.com/paper/2105.04319/) (2021)
- [A Unified Convergence Theorem for Stochastic Optimization Methods](https://mlsanity.com/paper/2206.03907/) (2022)
- [Lifted Bregman Training of Neural Networks](https://mlsanity.com/paper/2208.08772/) (2022)
- [Stochastic Zeroth order Descent with Structured Directions](https://mlsanity.com/paper/2206.05124/) (2022)
- [Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks](https://mlsanity.com/paper/2404.18769/) (2024)
- [A Normal Map-Based Proximal Stochastic Gradient Method: Convergence and Identification Properties](https://mlsanity.com/paper/2305.05828/) (2023)
- [Universal Regular Conditional Distributions](https://mlsanity.com/paper/2105.07743/) (2021)
- [A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization](https://mlsanity.com/paper/2312.01047/) (2023)
- [Sparse Training of Neural Networks based on Multilevel Mirror Descent](https://mlsanity.com/paper/2602.03535/) (2026)
- [Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates](https://mlsanity.com/paper/2403.11687/) (2024)
- [Sharper Bounds for Proximal Gradient Algorithms with Errors](https://mlsanity.com/paper/2203.02204/) (2022)
- [Iterative regularization in classification via hinge loss diagonal descent](https://mlsanity.com/paper/2212.12675/) (2022)
- [Stochastic First-Order Methods with Non-smooth and Non-Euclidean Proximal Terms for Nonconvex High-Dimensional Stochastic Optimization](https://mlsanity.com/paper/2406.19475/) (2024)
- [Proximal Iteration for Nonlinear Adaptive Lasso](https://mlsanity.com/paper/2412.05726/) (2024)
- [Learning Theory of the SVRG: Generalization and Convergence Analysis](https://mlsanity.com/paper/2605.28513/) (2026)

## Abstract (arXiv)

We prove novel convergence results for a stochastic proximal gradient algorithm suitable for solving a large class of convex optimization problems, where a convex objective function is given by the sum of a smooth and a possibly non-smooth component. We consider the iterates convergence and derive O(1/n) non asymptotic bounds in expectation in the strongly convex case, as well as almost sure convergence results under weaker assumptions. Our approach allows to avoid averaging and weaken boundedness assumptions which are often considered in theoretical studies and might not be satisfied in practice.