mlsanityData to 7 Oct 2026

Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function

arXiv 2112.14949 ↗First posted 30 Dec 2021math.OC cs.LG cs.SYcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper, we focus on the decentralized optimization problem over the Stiefel manifold, which is defined on a connected network of d agents. The objective is an average of d local functions, and each function is privately held by an agent and encodes its data. The agents can only communicate with their neighbors in a collaborative effort to solve this problem. In existing methods, multiple rounds of communications are required to guarantee the convergence, giving rise to high communication costs. In contrast, this paper proposes a decentralized algorithm, called DESTINY, which only invokes a single round of communications per iteration. DESTINY combines gradient tracking techniques with a novel approximate augmented Lagrangian function. The global convergence to stationary points is rigorously established. Comprehensive numerical experiments demonstrate that DESTINY has a strong potential to deliver a cutting-edge performance in solving a variety of testing problems.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 24 of its 60 references that are in the map. First posted on arXiv 30 Dec 2021.

Builds on

24 of 60 references.

References · 24

  1. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  2. 2014EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimizationcited by 160
  3. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  4. 2016Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphscited by 149
  5. 2016NEXT: In-Network Nonconvex Optimizationcited by 107
  6. 2017Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networkscited by 92
  7. 2017On orthogonality and learning recurrent networks with long term dependenciescited by 82
  8. 2016On Nonconvex Decentralized Gradient Descentcited by 42
  9. 2021Decentralized Riemannian Gradient Descent on the Stiefel Manifoldcited by 33
  10. 2021Accelerating Gossip SGD with Periodic Global Averagingcited by 25
  11. 2018Distributed Nonconvex Constrained Optimization over Time-Varying Digraphscited by 23
  12. 2014Multi-Agent Distributed Optimization via Inexact Consensus ADMMcited by 21
12 more
  1. 2021DeEPCA: Decentralized Exact PCA with Linear Convergence Ratecited by 18
  2. 2018Parallelizable Algorithms for Optimization Problems with Orthogonality Constraintscited by 16
  3. 2013A Framework of Constraint Preserving Update Schemes for Optimization on Stiefel Manifoldcited by 13
  4. 2021A Linearly Convergent Algorithm for Distributed Principal Component Analysiscited by 8
  5. 2021A Penalty-free Infeasible Approach for a Class of Nonsmooth Optimization Problems over the Stiefel Manifoldcited by 8
  6. 2018Structured Quasi-Newton Methods for Optimization with Orthogonality Constraintscited by 7
  7. 2018Second-order Guarantees of Distributed Gradient Algorithmscited by 5
  8. 2021Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Functioncited by 5
  9. 2020Multipliers Correction Methods for Optimization Problems over the Stiefel Manifoldcited by 4
  10. 2014A Dai-Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditionscited by 3
  11. 2021A Communication-Efficient and Privacy-Aware Distributed Algorithm for Sparse PCAcited by 3
  12. 2021FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysiscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 3; 2025: 2; 2026: 1.

11 papers.

  1. 2023Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifoldcited by 7
  2. 2024Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Datacited by 5
  3. 2022Variance reduced stochastic optimization over directed graphs with row and column stochastic weightscited by 2
  4. 2023Decentralized Riemannian natural gradient methods with Kronecker-product approximationscited by 1
  5. 2023Decentralized Weakly Convex Optimization Over the Stiefel Manifoldcited by 1
  6. 2024Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functionscited by 1
  7. 2025Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Trackingcited by 1
  8. 2022Decentralized Complete Dictionary Learning via ℓ4-Norm Maximizationcited by 0
  9. 2024A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraintscited by 0
  10. 2025A Support-Set Algorithm for Optimization Problems with Nonnegative and Orthogonal Constraintscited by 0
  11. 2026A second-order method landing on the Stiefel manifold via Newtonunicodex2013Schulz iterationcited by 0