# Optimization Techniques on Riemannian Manifolds

arXiv: https://arxiv.org/abs/1407.5965 · HTML: https://mlsanity.com/paper/1407.5965/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2014.

## Built on by (most cited first, 27 of 27)

- [Hyperbolic Fine-Tuning for Large Language Models](https://mlsanity.com/paper/2410.04010/) (2024)
- [A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold](https://mlsanity.com/paper/2302.08210/) (2023)
- [Momentum Stiefel Optimizer, with Applications to Suitably-Orthogonal Attention, and Optimal Transport](https://mlsanity.com/paper/2205.14173/) (2022)
- [O-ViT: Orthogonal Vision Transformer](https://mlsanity.com/paper/2201.12133/) (2022)
- [Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold](https://mlsanity.com/paper/2308.10547/) (2023)
- [What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold](https://mlsanity.com/paper/2505.11128/) (2025)
- [Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation](https://mlsanity.com/paper/2605.12492/) (2026)
- [tensorflow-riemopt: A Library for Optimization on Riemannian Manifolds](https://mlsanity.com/paper/2105.13921/) (2021)
- [Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs](https://mlsanity.com/paper/2205.08385/) (2022)
- [Manifold Free Riemannian Optimization](https://mlsanity.com/paper/2209.03269/) (2022)
- [Iso-Riemannian Optimization on Learned Data Manifolds](https://mlsanity.com/paper/2510.21033/) (2025)
- [Understanding Riemannian Acceleration via a Proximal Extragradient Framework](https://mlsanity.com/paper/2111.02763/) (2021)
- [Category-Level Object Shape and Pose Estimation in Less Than a Millisecond](https://mlsanity.com/paper/2509.18979/) (2025)
- [On the Benefits of Weight Normalization for Overparameterized Matrix Sensing](https://mlsanity.com/paper/2510.01175/) (2025)
- [Faster Randomized Methods for Orthogonality Constrained Problems](https://mlsanity.com/paper/2106.12060/) (2021)
- [A Survey on Distributed Online Optimization and Game](https://mlsanity.com/paper/2205.00473/) (2022)
- [Riemannian Stochastic Gradient Method for Nested Composition Optimization](https://mlsanity.com/paper/2207.09350/) (2022)
- [Synthetic Principal Component Design: Fast Covariate Balancing with Synthetic Controls](https://mlsanity.com/paper/2211.15241/) (2022)
- [A diffusion-map-based algorithm for gradient computation on manifolds and applications](https://mlsanity.com/paper/2108.06988/) (2021)
- [Generalised Spherical Text Embedding](https://mlsanity.com/paper/2211.16801/) (2022)
- [Extrinsic Bayesian Optimizations on Manifolds](https://mlsanity.com/paper/2212.13886/) (2022)
- [Riemannian Geometric-based Meta Learning](https://mlsanity.com/paper/2503.10993/) (2025)
- [Hadamard-Riemannian Optimization for Margin-Variance Ensemble](https://mlsanity.com/paper/2509.10189/) (2025)
- [Learning Topology-Driven Multi-Subspace Fusion for Grassmannian Deep Network](https://mlsanity.com/paper/2511.08628/) (2025)
- [Minimization of Functions on Dually Flat Spaces Using Geodesic Descent Based on Dual Connections](https://mlsanity.com/paper/2512.09358/) (2025)
- [Riemannian Zeroth-Order Gradient Estimation with Structure-Preserving Metrics for Geodesically Incomplete Manifolds](https://mlsanity.com/paper/2601.08039/) (2026)
- [Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamics](https://mlsanity.com/paper/2606.31390/) (2026)

## Abstract (arXiv)

The techniques and analysis presented in this paper provide new methods to solve optimization problems posed on Riemannian manifolds. A new point of view is offered for the solution of constrained optimization problems. Some classical optimization techniques on Euclidean space are generalized to Riemannian manifolds. Several algorithms are presented and their convergence properties are analyzed employing the Riemannian structure of the manifold. Specifically, two apparently new algorithms, which can be thought of as Newton's method and the conjugate gradient method on Riemannian manifolds, are presented and shown to possess, respectively, quadratic and superlinear convergence. Examples of each method on certain Riemannian manifolds are given with the results of numerical experiments. Rayleigh's quotient defined on the sphere is one example. It is shown that Newton's method applied to this function converges cubically, and that the Rayleigh quotient iteration is an efficient approximation of Newton's method. The Riemannian version of the conjugate gradient method applied to this function gives a new algorithm for finding the eigenvectors corresponding to the extreme eigenvalues of a symmetric matrix. Another example arises from extremizing the function mathoprm tr Thetascriptscriptstylerm TQThetaN on the special orthogonal group. In a similar example, it is shown that Newton's method applied to the sum of the squares of the off-diagonal entries of a symmetric matrix converges cubically.