mlsanityData to 14 Oct 2026

On the SAGA algorithm with decreasing step

arXiv 2410.03760 ↗First posted 2 Oct 2024math.OC cs.LG math.PRcited by 1

Not on the map. Why?

Abstract (arXiv)

Stochastic optimization naturally appear in many application areas, including machine learning. Our goal is to go further in the analysis of the Stochastic Average Gradient Accelerated (SAGA) algorithm. To achieve this, we introduce a new λ-SAGA algorithm which interpolates between the Stochastic Gradient Descent (λ=0) and the SAGA algorithm (λ=1). Firstly, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated to the objective function. Secondly, we establish a central limit theorem for the λ-SAGA algorithm. Finally, we provide the non-asymptotic mathbbLp rates of convergence.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 36 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

17 of 36 references.

References · 17

  1. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  2. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
  3. 2013Minimizing Finite Sums with the Stochastic Average Gradientcited by 164
  4. 2016Katyusha: The First Direct Acceleration of Stochastic Gradient Methodscited by 135
  5. 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 95
  6. 2020Variance-Reduced Methods for Machine Learningcited by 79
  7. 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
  8. 2016Stochastic Variance Reduction Methods for Saddle-Point Problemscited by 60
  9. 2022On Almost Sure Convergence Rates of Stochastic Gradient Methodscited by 35
  10. 2013Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regressioncited by 32
  11. 2018On the Theory of Variance Reduction for Stochastic Gradient Monte Carlocited by 19
  12. 2016A Simple Practical Accelerated Method for Finite Sumscited by 18
5 more
  1. 2021Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methodscited by 14
  2. 2018Local Convergence Properties of SAGA/Prox-SVRG and Accelerationcited by 10
  3. 2019SAGA with Arbitrary Samplingcited by 10
  4. 2017Tracking the gradients using the Hessian: A new look at variance reducing stochastic methodscited by 5
  5. 2021SGD with Coordinate Sampling: Theory and Practicecited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025A stochastic gradient descent algorithm with random search directionscited by 0