mlsanityData to 7 Oct 2026

Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds

arXiv 1605.07147 ↗First posted 23 May 2016math.OC cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

We study optimization of finite sums of geodesically smooth functions on Riemannian manifolds. Although variance reduction techniques for optimizing finite-sums have witnessed tremendous attention in the recent years, existing work is limited to vector space problems. We introduce Riemannian SVRG (RSVRG), a new variance reduced Riemannian optimization method. We analyze RSVRG for both geodesically convex and nonconvex (smooth) functions. Our analysis reveals that RSVRG inherits advantages of the usual SVRG method, but with factors depending on curvature of the manifold that influence its convergence. To our knowledge, RSVRG is the first provably fast stochastic Riemannian method. Moreover, our paper presents the first non-asymptotic complexity analysis (novel even for the batch setting) for nonconvex Riemannian optimization. Our results have several implications; for instance, they offer a Riemannian perspective on variance reduced PCA, which promises a short, transparent convergence analysis.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 17; 2023: 11; 2024: 11; 2025: 7; 2026: 9.

64 papers.

Sort:

By area:

  1. 2021Hyperbolic Busemann Learning with Ideal Prototypescited by 40
  2. 2021On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometrycited by 32
  3. 2021Projective Manifold Gradient Layer for Deep Rotation Regressioncited by 31
  4. 2023Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraintscited by 18
  5. 2025Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometriescited by 15
  6. 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
  7. 2022Federated Learning on Riemannian Manifoldscited by 13
  8. 2021On Riemannian Stochastic Approximation Schemes with Fixed Step-Sizecited by 12
  9. 2022Riemannian Hamiltonian methods for min-max optimization on manifoldscited by 12
  10. 2022Differentially private Riemannian optimizationcited by 12
  11. 2026Mano: Restriking Manifold Optimization for LLM Trainingcited by 12
  12. 2022Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processingcited by 10
38 more
  1. 2022Momentum Stiefel Optimizer, with Applications to Suitably-Orthogonal Attention, and Optimal Transportcited by 10
  2. 2022First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spacescited by 10
  3. 2022Sion's Minimax Theorem in Geodesic Metric Spaces and a Riemannian Extragradient Algorithmcited by 9
  4. 2022The Flag Median and FlagIRLScited by 9
  5. 2023Decentralized Riemannian Algorithm for Nonconvex Minimax Problemscited by 7
  6. 2023Particle-based Online Bayesian Samplingcited by 7
  7. 2023Zeroth-order Riemannian Averaging Stochastic Approximation Algorithmscited by 7
  8. 2024Optimization without Retraction on the Random Generalized Stiefel Manifoldcited by 7
  9. 2021From the Greene--Wu Convolution to Gradient Estimation over Riemannian Manifoldscited by 6
  10. 2022Riemannian Natural Gradient Methodscited by 5
  11. 2022Riemannian accelerated gradient methods via extrapolationcited by 5
  12. 2023The Fisher-Rao geometry of CES distributionscited by 5
  13. 2024Federated Learning on Riemannian Manifolds with Differential Privacycited by 5
  14. 2021No-go Theorem for Acceleration in the Hyperbolic Planecited by 4
  15. 2022Accelerated Riemannian Optimization: Handling Constraints with a Prox to Bound Geometric Penaltiescited by 4
  16. 2023Accelerated Methods for Riemannian Min-Max Optimization Ensuring Bounded Geometric Penaltiescited by 4
  17. 2024Riemannian coordinate descent algorithms on matrix manifoldscited by 4
  18. 2024Riemannian Federated Learning via Averaging Gradient Streamscited by 4
  19. 2021Understanding Riemannian Acceleration via a Proximal Extragradient Frameworkcited by 3
  20. 2022Hierarchical Symbolic Reasoning in Hyperbolic Space for Deep Discriminative Modelscited by 3
  21. 2024A Framework for Bilevel Optimization on Riemannian Manifoldscited by 3
  22. 2025Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Methodcited by 3
  23. 2025Online Optimization on Hadamard Manifolds: Curvature Independent Regret Bounds on Horospherically Convex Objectivescited by 3
  24. 2021rSVDdpd: A Robust Scalable Video Surveillance Background Modelling Algorithmcited by 2
  25. 2022A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraintscited by 2
  26. 2023Convergence of variational Monte Carlo simulation and scale-invariant pre-trainingcited by 2
  27. 2023Low-complexity subspace-descent over symmetric positive definite manifoldcited by 2
  28. 2023Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computationcited by 2
  29. 2024Stochastic Modified Flows for Riemannian Stochastic Gradient Descentcited by 2
  30. 2025Hyperbolic Large Language Modelscited by 2
  31. 2022On a class of geodesically convex optimization problems solved via Euclidean MM methodscited by 1
  32. 2022Riemannian Stochastic Gradient Method for Nested Composition Optimizationcited by 1
  33. 2022Rieoptax: Riemannian Optimization in JAXcited by 1
  34. 2023Constrained Empirical Risk Minimization: Theory and Practicecited by 1
  35. 2024Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reductioncited by 1
  36. 2024Learning-Rate-Free Stochastic Optimization over Riemannian Manifoldscited by 1
  37. 2024Disciplined Geodesically Convex Programmingcited by 1
  38. 2024Structured Regularization for Constrained Optimization on the SPD Manifoldcited by 1

50 most cited of 64 All 64 (JSON)