# On the Local Linear Rate of Consensus on the Stiefel Manifold

arXiv: https://arxiv.org/abs/2101.09346 · HTML: https://mlsanity.com/paper/2101.09346/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 76 references that are in the map.
- First posted on arXiv 22 Jan 2021.

## Builds on (most cited first, 24 of 24)

- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization](https://mlsanity.com/paper/1404.6264/) (2014)
- [Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphs](https://mlsanity.com/paper/1607.03218/) (2016)
- [Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization](https://mlsanity.com/paper/1709.08765/) (2017)
- [First-order Methods for Geodesically Convex Optimization](https://mlsanity.com/paper/1602.06053/) (2016)
- [On orthogonality and learning recurrent networks with long term dependencies](https://mlsanity.com/paper/1702.00071/) (2017)
- [Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution](https://mlsanity.com/paper/1711.10467/) (2017)
- [Global rates of convergence for nonconvex optimization on manifolds](https://mlsanity.com/paper/1605.08101/) (2016)
- [Error bounds, quadratic growth, and linear convergence of proximal methods](https://mlsanity.com/paper/1602.06661/) (2016)
- [Near-optimal bounds for phase synchronization](https://mlsanity.com/paper/1703.06605/) (2017)
- [Nonconvex phase synchronization](https://mlsanity.com/paper/1601.06114/) (2016)
- [Gradient methods for convex minimization: better rates under weaker conditions](https://mlsanity.com/paper/1303.4645/) (2013)
- [Balancing Communication and Computation in Distributed Optimization](https://mlsanity.com/paper/1709.02999/) (2017)
- [On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronization](https://mlsanity.com/paper/1603.00211/) (2016)
- [Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Data](https://mlsanity.com/paper/1412.7839/) (2014)
- [Convergence results for projected line-search methods on varieties of low-rank matrices via Lojasiewicz inequality](https://mlsanity.com/paper/1402.5284/) (2014)
- [On the Quadratic Convergence of the Cubic Regularization Method under a Local Error Bound Condition](https://mlsanity.com/paper/1801.09387/) (2018)
- [High-dimensional Kuramoto models on Stiefel manifolds synchronize complex networks almost globally](https://mlsanity.com/paper/1807.10233/) (2018)
- [Synchronization on Riemannian manifolds: Multiply connected implies multistable](https://mlsanity.com/paper/1906.07452/) (2019)
- [Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods](https://mlsanity.com/paper/1911.05047/) (2019)
- [Distributed Estimation of Graph Spectrum](https://mlsanity.com/paper/1503.08192/) (2015)
- [Almost global consensus on the n-sphere](https://mlsanity.com/paper/1609.04885/) (2016)
- [A Geometric Obstruction to Almost Global Synchronization on Riemannian Manifolds](https://mlsanity.com/paper/1808.00862/) (2018)

## Built on by (most cited first, 8 of 8)

- [Personalized PCA: Decoupling Shared and Unique Features](https://mlsanity.com/paper/2207.08041/) (2022)
- [Federated Learning on Riemannian Manifolds](https://mlsanity.com/paper/2206.05668/) (2022)
- [Decentralized Online Riemannian Optimization Beyond Hadamard Manifolds](https://mlsanity.com/paper/2509.07779/) (2025)
- [Decentralized Riemannian natural gradient methods with Kronecker-product approximations](https://mlsanity.com/paper/2303.09611/) (2023)
- [Decentralized Weakly Convex Optimization Over the Stiefel Manifold](https://mlsanity.com/paper/2303.17779/) (2023)
- [Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Tracking](https://mlsanity.com/paper/2506.07351/) (2025)
- [Decentralized Online Riemannian Optimization for Strongly Geodesically Convex Functions](https://mlsanity.com/paper/2607.20316/) (2026)
- [On the Estimation Performance of Generalized Power Method for Heteroscedastic Probabilistic PCA](https://mlsanity.com/paper/2312.03438/) (2023)

## Abstract (arXiv)

We study the convergence properties of Riemannian gradient method for solving the consensus problem (for an undirected connected graph) over the Stiefel manifold. The Stiefel manifold is a non-convex set and the standard notion of averaging in the Euclidean space does not work for this problem. We propose Distributed Riemannian Consensus on Stiefel Manifold (DRCS) and prove that it enjoys a local linear convergence rate to global consensus. More importantly, this local rate asymptotically scales with the second largest singular value of the communication matrix, which is on par with the well-known rate in the Euclidean space. To the best of our knowledge, this is the first work showing the equality of the two rates. The main technical challenges include (i) developing a Riemannian restricted secant inequality for convergence analysis, and (ii) to identify the conditions (e.g., suitable step-size and initialization) under which the algorithm always stays in the local region.