# A general framework of Riemannian adaptive optimization methods with a convergence analysis

arXiv: https://arxiv.org/abs/2409.00859 · HTML: https://mlsanity.com/paper/2409.00859/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2024.

## Built on by (most cited first, 5 of 5)

- [Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum](https://mlsanity.com/paper/2501.08883/) (2025)
- [Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size](https://mlsanity.com/paper/2501.18164/) (2025)
- [Efficient Generative Modeling with Unitary Matrix Product States Using Riemannian Optimization](https://mlsanity.com/paper/2603.12026/) (2026)
- [Online Supervised Dimension Reduction with Random Features: Diagnostics and Computational Trade-offs](https://mlsanity.com/paper/2609.20454/) (2026)
- [Stiefel-AdamW: Geometry-Aware AdamW for Linear Factorization Blocks](https://mlsanity.com/paper/2609.21039/) (2026)

## Abstract (arXiv)

This paper proposes a general framework of Riemannian adaptive optimization methods. The framework encapsulates several stochastic optimization algorithms on Riemannian manifolds and incorporates the mini-batch strategy that is often used in deep learning. Within this framework, we also propose AMSGrad on embedded submanifolds of Euclidean space. Moreover, we give convergence analyses valid for both a constant and a diminishing step size. Our analyses also reveal the relationship between the convergence rate and mini-batch size. In numerical experiments, we applied the proposed algorithm to principal component analysis and the low-rank matrix completion problem, which can be considered to be Riemannian optimization problems. Python implementations of the methods used in the numerical experiments are available at https://github.com/iiduka-researches/202408-adaptive.