mlsanityData to 7 Oct 2026

Averaging Stochastic Gradient Descent on Riemannian Manifolds

arXiv 1802.09128 ↗First posted 26 Feb 2018cs.LG math.OC stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

We consider the minimization of a function defined on a Riemannian manifold M accessible only through unbiased estimates of its gradients. We develop a geometric framework to transform a sequence of slowly converging iterates generated from stochastic gradient descent (SGD) on M to an averaged iterate sequence with a robust and fast O(1/n) convergence rate. We then present an application of our framework to geodesically-strongly-convex (and possibly Euclidean non-convex) problems. Finally, we demonstrate how these ideas apply to the case of streaming k-PCA, where we show how to accelerate the slow rate of the randomized power method (without requiring knowledge of the eigengap) into a robust algorithm achieving the optimal rate of convergence.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 7; 2024: 9; 2025: 5; 2026: 4.

39 papers.

Sort:

  1. 2021Fast and accurate optimization on the orthogonal manifold without retractioncited by 35
  2. 2021Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variancecited by 34
  3. 2024NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priorscited by 27
  4. 2022Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformercited by 22
  5. 2024Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codescited by 19
  6. 2023Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraintscited by 18
  7. 2022Optimal variance-reduced stochastic approximation in Banach spacescited by 14
  8. 2021On Riemannian Stochastic Approximation Schemes with Fixed Step-Sizecited by 12
  9. 2022First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spacescited by 10
  10. 2022Sion's Minimax Theorem in Geodesic Metric Spaces and a Riemannian Extragradient Algorithmcited by 9
  11. 2023Learning Rate Schedules in the Presence of Distribution Shiftcited by 7
  12. 2023Learning Optimal Power Flow Value Functions with Input-Convex Neural Networkscited by 7
27 more
  1. 2022Riemannian accelerated gradient methods via extrapolationcited by 5
  2. 2025Riemannian Optimization on Relaxed Indicator Matrix Manifoldcited by 5
  3. 2021No-go Theorem for Acceleration in the Hyperbolic Planecited by 4
  4. 2022Accelerated Riemannian Optimization: Handling Constraints with a Prox to Bound Geometric Penaltiescited by 4
  5. 2023Fairness Uncertainty Quantification: How certain are you that the model is fair?cited by 4
  6. 2024Riemannian coordinate descent algorithms on matrix manifoldscited by 4
  7. 2024Riemannian Federated Learning via Averaging Gradient Streamscited by 4
  8. 2024A quantitative Robbins-Siegmund theoremcited by 4
  9. 2022Riemannian stochastic approximation algorithmscited by 2
  10. 2023Convergence of variational Monte Carlo simulation and scale-invariant pre-trainingcited by 2
  11. 2023Low-complexity subspace-descent over symmetric positive definite manifoldcited by 2
  12. 2023Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computationcited by 2
  13. 2024Stochastic Modified Flows for Riemannian Stochastic Gradient Descentcited by 2
  14. 2025Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Sizecited by 2
  15. 2025A Geometric Analysis of PCAcited by 2
  16. 2021Towards Modeling and Resolving Singular Parameter Spaces using Stratifoldscited by 1
  17. 2022Riemannian Stochastic Gradient Method for Nested Composition Optimizationcited by 1
  18. 2024Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reductioncited by 1
  19. 2024Quantitative Convergences of Lie Group Momentum Optimizerscited by 1
  20. 2025An abstract effective convergence theorem for stochastic processes, with applications to stochastic approximationcited by 1
  21. 2021A diffusion-map-based algorithm for gradient computation on manifolds and applicationscited by 0
  22. 2024Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsizecited by 0
  23. 2025Online Learning of Functional Principal Component Analysis for Multidimensional Functional Datacited by 0
  24. 2026Riemannian Dueling Optimizationcited by 0
  25. 2026Mirror Descent on Riemannian Manifoldscited by 0
  26. 2026Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGDcited by 0
  27. 2026Locally Private Inference for Riemannian Stochastic Optimizationcited by 0