# Riemannian accelerated gradient methods via extrapolation

arXiv: https://arxiv.org/abs/2208.06619 · HTML: https://mlsanity.com/paper/2208.06619/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 77 references that are in the map.
- First posted on arXiv 13 Aug 2022.

## Builds on (most cited first, 30 of 34)

- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [Manopt, a Matlab toolbox for optimization on manifolds](https://mlsanity.com/paper/1308.5200/) (2013)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [First-order Methods for Geodesically Convex Optimization](https://mlsanity.com/paper/1602.06053/) (2016)
- [Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds](https://mlsanity.com/paper/1605.07147/) (2016)
- [Global rates of convergence for nonconvex optimization on manifolds](https://mlsanity.com/paper/1605.08101/) (2016)
- [Acceleration Methods](https://mlsanity.com/paper/2101.09545/) (2021)
- [Averaging Stochastic Gradient Descent on Riemannian Manifolds](https://mlsanity.com/paper/1802.09128/) (2018)
- [Escaping from saddle points on Riemannian manifolds](https://mlsanity.com/paper/1906.07355/) (2019)
- [A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimization](https://mlsanity.com/paper/1910.10782/) (2019)
- [Riemannian Dictionary Learning and Sparse Coding for Positive Definite Matrices](https://mlsanity.com/paper/1507.02772/) (2015)
- [From Nesterov's Estimate Sequence to Riemannian Acceleration](https://mlsanity.com/paper/2001.08876/) (2020)
- [Regularized Nonlinear Acceleration](https://mlsanity.com/paper/1606.04133/) (2016)
- [Adaptive regularization with cubics on manifolds](https://mlsanity.com/paper/1806.00065/) (2018)
- [An accelerated first-order method for non-convex optimization on manifolds](https://mlsanity.com/paper/2008.02252/) (2020)
- [Geometry-aware Domain Adaptation for Unsupervised Alignment of Word Embeddings](https://mlsanity.com/paper/2004.08243/) (2020)
- [Dictionary Learning and Sparse Coding on Grassmann Manifolds: An Extrinsic Solution](https://mlsanity.com/paper/1310.4891/) (2013)
- [Negative curvature obstructs acceleration for strongly geodesically convex optimization, even with exact first-order oracles](https://mlsanity.com/paper/2111.13263/) (2021)
- [Global Riemannian Acceleration in Hyperbolic and Spherical Spaces](https://mlsanity.com/paper/2012.03618/) (2020)
- [Accelerated Gradient Methods for Geodesically Convex Optimization: Tractable Algorithms and Convergence Analysis](https://mlsanity.com/paper/2202.02036/) (2022)
- [A dual framework for low-rank tensor completion](https://mlsanity.com/paper/1712.01193/) (2017)
- [Manifold optimization for non-linear optimal transport problems](https://mlsanity.com/paper/2103.00902/) (2021)
- [Rapid Mixing of Geodesic Walks on Manifolds with Positive Curvature](https://mlsanity.com/paper/1609.02901/) (2016)
- [Riemannian stochastic recursive momentum method for non-convex optimization](https://mlsanity.com/paper/2008.04555/) (2020)
- [Riemannian block SPD coupling manifold and its application to optimal transport](https://mlsanity.com/paper/2201.12933/) (2022)
- [Accelerated Optimization With Orthogonality Constraints](https://mlsanity.com/paper/1903.05204/) (2019)
- [A Variational Formulation of Accelerated Optimization on Riemannian Manifolds](https://mlsanity.com/paper/2101.06552/) (2021)
- [Nonlinear Acceleration of Momentum and Primal-Dual Algorithms](https://mlsanity.com/paper/1810.04539/) (2018)

## Built on by (most cited first, 5 of 5)

- [A Framework for Bilevel Optimization on Riemannian Manifolds](https://mlsanity.com/paper/2402.03883/) (2024)
- [Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method](https://mlsanity.com/paper/2505.12378/) (2025)
- [Rieoptax: Riemannian Optimization in JAX](https://mlsanity.com/paper/2210.04840/) (2022)
- [Function Extrapolation with Neural Networks and Its Application for Manifolds](https://mlsanity.com/paper/2405.10563/) (2024)
- [Quantum Optimization via Gradient-Based Hamiltonian Descent](https://mlsanity.com/paper/2505.14670/) (2025)

## Abstract (arXiv)

In this paper, we propose a simple acceleration scheme for Riemannian gradient methods by extrapolating iterates on manifolds. We show when the iterates are generated from Riemannian gradient descent method, the accelerated scheme achieves the optimal convergence rate asymptotically and is computationally more favorable than the recently proposed Riemannian Nesterov accelerated gradient methods. Our experiments verify the practical benefit of the novel acceleration strategy.