# On the Convergence of Stochastic Gradient Descent for Linear Inverse Problems in Banach Spaces

arXiv: https://arxiv.org/abs/2302.05197 · HTML: https://mlsanity.com/paper/2302.05197/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 60 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 10 of 10)

- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [SGD: General Analysis and Improved Rates](https://mlsanity.com/paper/1901.09401/) (2019)
- [Convex Optimization in Sums of Banach Spaces](https://mlsanity.com/paper/2104.13127/) (2021)
- [Linear convergence of the Randomized Sparse Kaczmarz Method](https://mlsanity.com/paper/1610.02889/) (2016)
- [On the Convergence of Stochastic Gradient Descent for Nonlinear Ill-Posed Problems](https://mlsanity.com/paper/1907.03132/) (2019)
- [On the Saturation Phenomenon of Stochastic Gradient Descent for Linear Inverse Problems](https://mlsanity.com/paper/2010.10916/) (2020)
- [Extended Randomized Kaczmarz Method for Sparse Least Squares and Impulsive Noise Problems](https://mlsanity.com/paper/2201.08620/) (2022)
- [Stochastic mirror descent method for linear ill-posed problems in Banach spaces](https://mlsanity.com/paper/2207.06584/) (2022)
- [Online learning in optical tomography: a stochastic approach](https://mlsanity.com/paper/1710.09521/) (2017)
- [Variational source conditions in Lp-spaces](https://mlsanity.com/paper/2007.13474/) (2020)

## Built on by (most cited first, 2 of 2)

- [Large Language Models in Education: Vision and Opportunities](https://mlsanity.com/paper/2311.13160/) (2023)
- [Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent](https://mlsanity.com/paper/2306.11589/) (2023)

## Abstract (arXiv)

In this work we consider stochastic gradient descent (SGD) for solving linear inverse problems in Banach spaces. SGD and its variants have been established as one of the most successful optimisation methods in machine learning, imaging and signal processing, etc. At each iteration SGD uses a single datum, or a small subset of data, resulting in highly scalable methods that are very attractive for large-scale inverse problems. Nonetheless, the theoretical analysis of SGD-based approaches for inverse problems has thus far been largely limited to Euclidean and Hilbert spaces. In this work we present a novel convergence analysis of SGD for linear inverse problems in general Banach spaces: we show the almost sure convergence of the iterates to the minimum norm solution and establish the regularising property for suitable a priori stopping criteria. Numerical results are also presented to illustrate features of the approach.