Randomized Kaczmarz with geometrically smoothed momentum
Not on the map. Why?
Abstract (arXiv)
This paper studies the effect of adding geometrically smoothed momentum to the randomized Kaczmarz algorithm, which is an instance of stochastic gradient descent on a linear least squares loss function. We prove a result about the expected error in the direction of singular vectors of the matrix defining the least squares loss. We present several numerical examples illustrating the utility of our result and pose several questions.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 14 of its 35 references that are in the map. First posted on arXiv 17 Jan 2024.
Builds on
14 of 35 references.
References · 14
- 2013Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithmcited by 132
- 2017Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methodscited by 49
- 2018On the insufficiency of existing momentum schemes for Stochastic Optimizationcited by 46
- 2022Sharp Analysis of Sketch-and-Project Methods via a Connection to Randomized Singular Value Decompositioncited by 17
- 2013An Accelerated Randomized Kaczmarz Algorithmcited by 12
- 2022On the fast convergence of minibatch heavy ball momentumcited by 11
- 2020Randomized Kaczmarz converges along small singular vectorscited by 9
- 2019Faster randomized block Kaczmarz algorithmscited by 5
- 2020Sampling Kaczmarz Motzkin Method for Linear Feasibility Problems: Generalization & Accelerationcited by 5
- 2023On adaptive stochastic heavy ball momentum for solving linear systemscited by 3
- 2022Randomized Douglas-Rachford methods for linear systems: Improved accuracy and efficiencycited by 2
- 2022An optimal scheduled learning rate for a randomized Kaczmarz algorithmcited by 1
Built on by
Citing papers per year: 2024: 1; 2025: 1; 2026: 1.
3 papers.