mlsanityData to 14 Oct 2026

Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds

arXiv 2406.02296 ↗First posted 4 Jun 2024cs.LG math.OCcited by 1

Not on the map. Why?

Abstract (arXiv)

In recent years, interest in gradient-based optimization over Riemannian manifolds has surged. However, a significant challenge lies in the reliance on hyperparameters, especially the learning rate, which requires meticulous tuning by practitioners to ensure convergence at a suitable rate. In this work, we introduce innovative learning-rate-free algorithms for stochastic optimization over Riemannian manifolds, eliminating the need for hand-tuning and providing a more robust and user-friendly approach. We establish high probability convergence guarantees that are optimal, up to logarithmic factors, compared to the best-known optimally tuned rate in the deterministic setting. Our approach is validated through numerical experiments, demonstrating competitive performance against learning-rate-dependent algorithms.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 57 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

28 of 57 references.

References · 28

  1. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  2. 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 206
  3. 2018Riemannian Adaptive Optimization Methodscited by 194
  4. 2016Coin Betting and Parameter-Free Online Learningcited by 133
  5. 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
  6. 2016Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiationcited by 94
  7. 2016First-order Methods for Geodesically Convex Optimizationcited by 93
  8. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  9. 2015Beyond Convexity: Stochastic Quasi-Convex Optimizationcited by 76
  10. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  11. 2016Global rates of convergence for nonconvex optimization on manifoldscited by 62
  12. 2022Making SGD Parameter-Freecited by 50
16 more
  1. 2014Unconstrained Online Linear Learning in Hilbert Spaces: Minimax Algorithms and Normal Approximationscited by 46
  2. 2017Riemannian approach to batch normalizationcited by 35
  3. 2023DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Methodcited by 35
  4. 2016The Power of Normalization: Faster Evasion of Saddle Pointscited by 29
  5. 2019A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimizationcited by 25
  6. 2015Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Methodcited by 24
  7. 2020From Nesterov's Estimate Sequence to Riemannian Accelerationcited by 24
  8. 2016Geometric Mean Metric Learningcited by 22
  9. 2017Convergence Rates for Deterministic and Stochastic Subgradient Methods Without Lipschitz Continuitycited by 19
  10. 2018Adaptive regularization with cubics on manifoldscited by 17
  11. 2018Towards Riemannian Accelerated Gradient Methodscited by 12
  12. 2018Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifoldscited by 12
  13. 2023Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Bettingcited by 11
  14. 2023Coin Sampling: Gradient-Based Bayesian Inference without Learning Ratescited by 8
  15. 2016Riemannian stochastic variance reduced gradient on Grassmann manifoldcited by 4
  16. 2023Learning Rate Free Sampling in Constrained Domainscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Stochastic Optimization of Tree Tensor Networkscited by 0