mlsanityData to 7 Oct 2026

On Riemannian Stochastic Approximation Schemes with Fixed Step-Size

arXiv 2102.07586 ↗First posted 15 Feb 2021stat.ML cs.LG math.PRcited by 12

Not on the map. Why?

Abstract (arXiv)

This paper studies fixed step-size stochastic approximation (SA) schemes, including stochastic gradient schemes, in a Riemannian framework. It is motivated by several applications, where geodesics can be computed explicitly, and their use accelerates crude Euclidean methods. A fixed step-size scheme defines a family of time-homogeneous Markov chains, parametrized by the step-size. Here, using this formulation, non-asymptotic performance bounds are derived, under Lyapunov conditions. Then, for any step-size, the corresponding Markov chain is proved to admit a unique stationary distribution, and to be geometrically ergodic. This result gives rise to a family of stationary distributions indexed by the step-size, which is further shown to converge to a Dirac measure, concentrated at the solution of the problem at hand, as the step-size goes to 0. Finally, the asymptotic rate of this convergence is established, through an asymptotic expansion of the bias, and a central limit theorem.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 17 of its 51 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

17 of 51 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learningcited by 150
  3. 2018Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptroncited by 143
  4. 2013Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithmcited by 132
  5. 2016First-order Methods for Geodesically Convex Optimizationcited by 92
  6. 2017Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chainscited by 84
  7. 2016Gradient Descent Learns Linear Dynamical Systemscited by 78
  8. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  9. 2016The non-convex Burer-Monteiro approach works on smooth semidefinite programscited by 44
  10. 2018Averaging Stochastic Gradient Descent on Riemannian Manifoldscited by 39
  11. 2019A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimizationcited by 25
  12. 2015Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Methodcited by 24
5 more
  1. 2017An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimizationcited by 20
  2. 2020Convergence Analysis of Riemannian Stochastic Approximation Schemescited by 5
  3. 2017Stochastic Primal-Dual Method on Riemannian Manifolds with Bounded Sectional Curvaturecited by 4
  4. 2020On the Effectiveness of Richardson Extrapolation in Machine Learningcited by 2
  5. 2020Variance reduction for Riemannian non-convex optimization with batch size adaptationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 4; 2024: 3; 2025: 1; 2026: 1.

12 papers.

  1. 2023Metropolis Sampling for Constrained Diffusion Modelscited by 28
  2. 2024Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolationcited by 17
  3. 2021Stationary Behavior of Constant Stepsize SGD Type Algorithms: An Asymptotic Characterizationcited by 10
  4. 2023Stochastic Methods in Variational Inequalities: Ergodicity, Bias and Refinementscited by 6
  5. 2023Backward error analysis and the qualitative behaviour of stochastic optimization algorithms: Application to stochastic coordinate descentcited by 5
  6. 2024Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SAcited by 3
  7. 2022Riemannian stochastic approximation algorithmscited by 2
  8. 2024Stochastic Modified Flows for Riemannian Stochastic Gradient Descentcited by 2
  9. 2025Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Sizecited by 2
  10. 2022The Symmetric Generalized Eigenvalue Problem as a Nash Equilibriumcited by 0
  11. 2023Tame Riemannian Stochastic Approximationcited by 0
  12. 2026Stochastic gradient descent with discontinuity across a manifoldcited by 0