mlsanityData to 7 Oct 2026

Decentralized Riemannian Gradient Descent on the Stiefel Manifold

arXiv 2102.07091 ↗First posted 14 Feb 2021math.OC cs.LG cs.SYcited by 33

Not on the map. Why?

Abstract (arXiv)

We consider a distributed non-convex optimization where a network of agents aims at minimizing a global function over the Stiefel manifold. The global function is represented as a finite sum of smooth local functions, where each local function is associated with one agent and agents communicate with each other over an undirected connected graph. The problem is non-convex as local functions are possibly non-convex (but smooth) and the Steifel manifold is a non-convex set. We present a decentralized Riemannian stochastic gradient method (DRSGD) with the convergence rate of O(1/√K) to a stationary point. To have exact convergence with constant stepsize, we also propose a decentralized Riemannian gradient tracking algorithm (DRGTA) with the convergence rate of O(1/K) to a stationary point. We use multi-step consensus to preserve the iteration in the local (consensus) region. DRGTA is the first decentralized algorithm with exact convergence for distributed optimization on Stiefel manifold.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 27 of its 55 references that are in the map. First posted on arXiv 14 Feb 2021.

Builds on

27 of 55 references.

References · 27

  1. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  2. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  3. 2014EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimizationcited by 160
  4. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  5. 2018Stochastic Gradient Push for Distributed Deep Learningcited by 152
  6. 2016Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphscited by 149
  7. 2017Network Topology and Communication-Computation Tradeoffs in Decentralized Optimizationcited by 119
  8. 2016NEXT: In-Network Nonconvex Optimizationcited by 107
  9. 2016First-order Methods for Geodesically Convex Optimizationcited by 92
  10. 2017Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networkscited by 92
  11. 2017On orthogonality and learning recurrent networks with long term dependenciescited by 82
  12. 2016Global rates of convergence for nonconvex optimization on manifoldscited by 61
15 more
  1. 2013On the Linear Convergence of the ADMM in Decentralized Consensus Optimizationcited by 57
  2. 2017Exact Diffusion for Distributed Optimization and Learning --- Part I: Algorithm Developmentcited by 50
  3. 2014Multi-Agent Distributed Optimization via Inexact Consensus ADMMcited by 21
  4. 2019Decentralized Proximal Gradient Algorithms with Linear Convergence Ratescited by 19
  5. 2021DeEPCA: Decentralized Exact PCA with Linear Convergence Ratecited by 18
  6. 2014Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Datacited by 15
  7. 2017Exact Diffusion for Distributed Optimization and Learning --- Part II: Convergence Analysiscited by 14
  8. 2015Distributed Linearized Alternating Direction Method of Multipliers for Composite Convex Consensus Optimizationcited by 13
  9. 2019Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: A Joint Gradient Estimation and Tracking Approachcited by 11
  10. 2021A Linearly Convergent Algorithm for Distributed Principal Component Analysiscited by 8
  11. 2017Distributed Optimization on Riemannian Manifolds for multi-agent networkscited by 5
  12. 2018High-dimensional Kuramoto models on Stiefel manifolds synchronize complex networks almost globallycited by 4
  13. 2019Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methodscited by 2
  14. 2020On the Divergence of Decentralized Non-Convex Optimizationcited by 2
  15. 2020On Distributed Non-convex Optimization: Projected Subgradient Method For Weakly Convex Problems in Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 5; 2024: 6; 2025: 7; 2026: 7.

33 papers.

Sort:

By area:

  1. 2022Personalized PCA: Decoupling Shared and Unique Featurescited by 17
  2. 2022Federated Learning on Riemannian Manifoldscited by 13
  3. 2021Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Functioncited by 11
  4. 2025Unified Analysis of Decentralized Gradient Descent: a Contraction Mapping Frameworkcited by 10
  5. 2023Decentralized Riemannian Algorithm for Nonconvex Minimax Problemscited by 7
  6. 2023Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifoldcited by 7
  7. 2024Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Datacited by 5
  8. 2024Stiefel Flow Matching for Moment-Constrained Structure Elucidationcited by 5
  9. 2025Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigationcited by 5
  10. 2021Distributed Principal Subspace Analysis for Partitioned Big Data: Algorithms, Analysis, and Implementationcited by 4
  11. 2025Decentralized Online Riemannian Optimization Beyond Hadamard Manifoldscited by 4
  12. 2021FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysiscited by 3
21 more
  1. 2025Online Optimization on Hadamard Manifolds: Curvature Independent Regret Bounds on Horospherically Convex Objectivescited by 3
  2. 2022A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraintscited by 2
  3. 2026Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steeringcited by 2
  4. 2022A Survey of Learning on Small Data: Generalization, Optimization, and Challengecited by 1
  5. 2023Decentralized Riemannian natural gradient methods with Kronecker-product approximationscited by 1
  6. 2023Decentralized Weakly Convex Optimization Over the Stiefel Manifoldcited by 1
  7. 2024ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learningcited by 1
  8. 2025Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Trackingcited by 1
  9. 2025Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approachcited by 1
  10. 2025Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifoldscited by 1
  11. 2026Decentralized Online Riemannian Optimization for Strongly Geodesically Convex Functionscited by 1
  12. 2022Decentralized Complete Dictionary Learning via ℓ4-Norm Maximizationcited by 0
  13. 2023Convex optimization over a probability simplexcited by 0
  14. 2024Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithmscited by 0
  15. 2024A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraintscited by 0
  16. 2024Rapid Grassmannian Averaging with Chebyshev Polynomialscited by 0
  17. 2026Improved Analysis of the Accelerated Noisy Power Method with Applications to Decentralized PCAcited by 0
  18. 2026Riemannian Stochastic Optimization for Sufficient Dimension Reductioncited by 0
  19. 2026Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamicscited by 0
  20. 2026Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuningcited by 0
  21. 2026Incremental Aggregation on the Grassmannian for Asynchronous Eigenspace Computationcited by 0