Decentralized Weakly Convex Optimization Over the Stiefel Manifold
Not on the map. Why?
Abstract (arXiv)
We focus on a class of non-smooth optimization problems over the Stiefel manifold in the decentralized setting, where a connected network of n agents cooperatively minimize a finite-sum objective function with each component being weakly convex in the ambient Euclidean space. Such optimization problems, albeit frequently encountered in applications, are quite challenging due to their non-smoothness and non-convexity. To tackle them, we propose an iterative method called the decentralized Riemannian subgradient method (DRSM). The global convergence and an iteration complexity of O(varepsilon-2 log2(varepsilon-1)) for forcing a natural stationarity measure below varepsilon are established via the powerful tool of proximal smoothness from variational analysis, which could be of independent interest. Besides, we show the local linear convergence of the DRSM using geometrically diminishing stepsizes when the problem at hand further possesses a sharpness property. Numerical experiments are conducted to corroborate our theoretical findings.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 76 references that are in the map. First posted on arXiv 31 Mar 2023.
Builds on
39 of 76 references.
References · 39
- 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
- 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
- 2013On the Convergence of Decentralized Gradient Descentcited by 158
- 2018Stochastic model-based minimization of weakly convex functionscited by 157
- 2018D2: Decentralized Training over Decentralized Datacited by 140
- 2016NEXT: In-Network Nonconvex Optimizationcited by 107
- 2016Global rates of convergence for nonconvex optimization on manifoldscited by 61
- 2021Exponential Graph is Provably Efficient for Decentralized Deep Trainingcited by 55
- 2016On Nonconvex Decentralized Gradient Descentcited by 42
- 2019A Brief Introduction to Manifold Optimizationcited by 35
- 2018Subgradient methods for sharp weakly convex functionscited by 33
- 2021Decentralized Riemannian Gradient Descent on the Stiefel Manifoldcited by 33
27 more
- 2018An Overview of Robust Subspace Recoverycited by 29
- 2015A Unified Approach to Error Bounds for Structured Convex Optimization Problemscited by 21
- 2020Understanding Notions of Stationarity in Non-Smooth Optimizationcited by 18
- 2021DeEPCA: Decentralized Exact PCA with Linear Convergence Ratecited by 18
- 2016On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase Synchronizationcited by 17
- 2021Convergence of Random Reshuffling Under The Kurdyka-Lojasiewicz Inequalitycited by 16
- 2014Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big, Distributed Datacited by 15
- 2020A Unified Approach to Synchronization Problems over Subgroups of the Orthogonal Groupcited by 15
- 2021Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Functioncited by 11
- 2014Dictionary Learning over Distributed Modelscited by 8
- 2021On the Local Linear Rate of Consensus on the Stiefel Manifoldcited by 8
- 2016A Family of Inexact SQA Methods for Non-Smooth Convex Minimization with Provable Convergence Guarantees Based on the Luo-Tseng Error Bound Propertycited by 7
- 2020Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learningcited by 6
- 2021Distributed Random Reshuffling over Networkscited by 6
- 2018Second-order Guarantees of Distributed Gradient Algorithmscited by 5
- 2018High-dimensional Kuramoto models on Stiefel manifolds synchronize complex networks almost globallycited by 4
- 2020Stochastic optimization over proximally smooth setscited by 4
- 2021Decentralized Composite Optimization in Stochastic Networks: A Dual Averaging Approach with Linear Convergencecited by 4
- 2021Orthogonal Group Synchronization with Incomplete Measurements: Error Bounds and Linear Convergence of the Generalized Power Methodcited by 4
- 2018Decentralized Dictionary Learning Over Time-Varying Digraphscited by 3
- 2019Asynchronous Decentralized Successive Convex Approximationcited by 3
- 2019Error bound conditions and convergence of optimization methods on smooth and proximally smooth manifoldscited by 3
- 2020Asynchronous Optimization over Graphs: Linear Convergence under Error Bound Conditionscited by 3
- 2013Non-Asymptotic Convergence Analysis of Inexact Gradient Methods for Machine Learning Without Strong Convexitycited by 2
- 2018Nonconvex weak sharp minima on Riemannian manifoldscited by 2
- 2017Distributed Robust Subspace Recoverycited by 1
- 2019Unique Sharp Local Minimum in ℓ1-minimization Complete Dictionary Learningcited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.