mlsanityData to 7 Oct 2026

On the Local Linear Rate of Consensus on the Stiefel Manifold

arXiv 2101.09346 ↗First posted 22 Jan 2021math.OC cs.LG cs.SYcited by 8

Not on the map. Why?

Abstract (arXiv)

We study the convergence properties of Riemannian gradient method for solving the consensus problem (for an undirected connected graph) over the Stiefel manifold. The Stiefel manifold is a non-convex set and the standard notion of averaging in the Euclidean space does not work for this problem. We propose Distributed Riemannian Consensus on Stiefel Manifold (DRCS) and prove that it enjoys a local linear convergence rate to global consensus. More importantly, this local rate asymptotically scales with the second largest singular value of the communication matrix, which is on par with the well-known rate in the Euclidean space. To the best of our knowledge, this is the first work showing the equality of the two rates. The main technical challenges include (i) developing a Riemannian restricted secant inequality for convergence analysis, and (ii) to identify the conditions (e.g., suitable step-size and initialization) under which the algorithm always stays in the local region.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 24 of its 76 references that are in the map. First posted on arXiv 22 Jan 2021.

Builds on

24 of 76 references.

References · 24

  1. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  2. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  3. 2014EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimizationcited by 160
  4. 2016Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphscited by 149
  5. 2017Network Topology and Communication-Computation Tradeoffs in Decentralized Optimizationcited by 119
  6. 2016First-order Methods for Geodesically Convex Optimizationcited by 92
  7. 2017On orthogonality and learning recurrent networks with long term dependenciescited by 82
  8. 2017Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolutioncited by 63
  9. 2016Global rates of convergence for nonconvex optimization on manifoldscited by 61
  10. 2016Error bounds, quadratic growth, and linear convergence of proximal methodscited by 59
  11. 2017Near-optimal bounds for phase synchronizationcited by 32
  12. 2016Nonconvex phase synchronizationcited by 30
12 more
  1. 2017Balancing Communication and Computation in Distributed Optimizationcited by 23
  2. 2013Gradient methods for convex minimization: better rates under weaker conditionscited by 21
  3. 2016On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronizationcited by 17
  4. 2014Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Datacited by 15
  5. 2014Convergence results for projected line-search methods on varieties of low-rank matrices via Lojasiewicz inequalitycited by 10
  6. 2018On the Quadratic Convergence of the Cubic Regularization Method under a Local Error Bound Conditioncited by 7
  7. 2018High-dimensional Kuramoto models on Stiefel manifolds synchronize complex networks almost globallycited by 4
  8. 2019Synchronization on Riemannian manifolds: Multiply connected implies multistablecited by 3
  9. 2019Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methodscited by 2
  10. 2015Distributed Estimation of Graph Spectrumcited by 1
  11. 2016Almost global consensus on the n-spherecited by 1
  12. 2018A Geometric Obstruction to Almost Global Synchronization on Riemannian Manifoldscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 0; 2025: 2; 2026: 1.

8 papers.

  1. 2022Personalized PCA: Decoupling Shared and Unique Featurescited by 17
  2. 2022Federated Learning on Riemannian Manifoldscited by 13
  3. 2025Decentralized Online Riemannian Optimization Beyond Hadamard Manifoldscited by 4
  4. 2023Decentralized Riemannian natural gradient methods with Kronecker-product approximationscited by 1
  5. 2023Decentralized Weakly Convex Optimization Over the Stiefel Manifoldcited by 1
  6. 2025Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Trackingcited by 1
  7. 2026Decentralized Online Riemannian Optimization for Strongly Geodesically Convex Functionscited by 1
  8. 2023On the Estimation Performance of Generalized Power Method for Heteroscedastic Probabilistic PCAcited by 0