mlsanityData to 14 Oct 2026

Incremental Aggregation on the Grassmannian for Asynchronous Eigenspace Computation

arXiv 2608.04406 ↗First posted 5 Aug 2026math.OC cs.NA eess.SPcited by 0

Not on the map. Why?

Abstract (arXiv)

We study asynchronous optimization for finite-sum eigenspace computation in heterogeneous distributed systems. The theoretical foundations for asynchronous eigenspace computation remain scarce, with existing approaches offering limited coverage of dynamics directly on the Grassmannian under stale information. In this paper, we propose a Grassmannian incremental aggregation method that refreshes only arriving components and reuses cached gradients, retaining low per-update cost without global synchronization. The method employs an extrinsic polar update that preserves the intrinsic subspace geometry without requiring parallel transport of stale tangent vectors. Our analysis establishes a tight angle-dependent gradient-dominance characterization of the objective and a basin-invariance property for stale aggregated updates. These yield two-phase linear convergence, comprising an explicit broad-basin regime and a sharper local regime, with constants controlled by component spectral spreads. Experiments on serial and distributed PCA demonstrate improved sample efficiency and wall-clock convergence over representative baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 58 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

31 of 58 references.

References · 31

  1. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 463
  2. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 238
  3. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  4. 2013The Noisy Power Method: A Meta Algorithm with Applicationscited by 58
  5. 2020Advances in Asynchronous Parallel and Distributed Optimizationcited by 40
  6. 2019A Brief Introduction to Manifold Optimizationcited by 35
  7. 2021Decentralized Riemannian Gradient Descent on the Stiefel Manifoldcited by 33
  8. 2016First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Ratecited by 32
  9. 2016Streaming PCA: Matching Matrix Bernstein and Near-Optimal Finite Sample Guarantees for Oja's Algorithmcited by 29
  10. 2017Accelerated Stochastic Power Iterationcited by 22
  11. 2013Memory Limited, Streaming PCAcited by 21
  12. 2015On the Convergence Rate of Incremental Aggregated Gradient Algorithmscited by 21
19 more
  1. 2021Streaming k-PCA: Efficient guarantees for Oja's algorithm, beyond rank-one updatescited by 21
  2. 2014A Stochastic PCA and SVD Algorithm with an Exponential Convergence Ratecited by 18
  3. 2016Faster Eigenvector Computation via Shift-and-Invert Preconditioningcited by 18
  4. 2021DeEPCA: Decentralized Exact PCA with Linear Convergence Ratecited by 18
  5. 2015The Fast Convergence of Incremental PCAcited by 15
  6. 2019Exponentially convergent stochastic k-PCA without variance reductioncited by 12
  7. 2020Communication-Efficient Distributed SVD via Local Power Iterationscited by 12
  8. 2016Near-Optimal Stochastic Approximation for Online Principal Component Estimationcited by 11
  9. 2016Global Convergence Rate of Proximal Incremental Aggregated Gradient Methodscited by 11
  10. 2015Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and Convexitycited by 9
  11. 2016Analysis and Implementation of an Asynchronous Optimization Algorithm for the Parameter Servercited by 7
  12. 2021Distributed Principal Component Analysis with Limited Communicationcited by 6
  13. 2017Proximal-Like Incremental Aggregated Gradient Method with Linear Convergence under Bregman Distance Growth Conditionscited by 5
  14. 2023Linear Speedup of Incremental Aggregated Gradient Methods on Streaming Datacited by 5
  15. 2021FedPower: Privacy-Preserving Distributed Eigenspace Estimationcited by 4
  16. 2021Distributed Principal Subspace Analysis for Partitioned Big Data: Algorithms, Analysis, and Implementationcited by 4
  17. 2022Geodesic Convexity of the Symmetric Eigenvalue Problem and Convergence of Riemannian Steepest Descentcited by 2
  18. 2018Nonconvex Proximal Incremental Aggregated Gradient Method with Linear Convergencecited by 1
  19. 2021Distributed Banach-Picard Iteration: Application to Distributed EM and Distributed PCAcited by 1

Built on by

Not cited yet.