mlsanityData to 7 Oct 2026

A Stochastic PCA and SVD Algorithm with an Exponential Convergence Rate

arXiv 1409.2848 ↗First posted 9 Sep 2014cs.LG cs.NA math.NAcited by 18

Not on the map. Why?

Abstract (arXiv)

We describe and analyze a simple algorithm for principal component analysis and singular value decomposition, VR-PCA, which uses computationally cheap stochastic iterations, yet converges exponentially fast to the optimal solution. In contrast, existing algorithms suffer either from slow convergence, or computationally intensive iterations whose runtime scales with the data size. The algorithm builds on a recent variance-reduced stochastic gradient technique, which was previously analyzed for strongly convex optimization, whereas here we apply it to an inherently non-convex problem, using a very different analysis.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Sep 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 3; 2023: 3; 2024: 1; 2025: 2; 2026: 1.

18 papers.

Sort:

  1. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  2. 2021Fast and Efficient MMD-based Fair PCA via Optimization over Stiefel Manifoldcited by 18
  3. 2021Learning a Single Neuron with Bias Using Gradient Descentcited by 17
  4. 2023The Edge of Orthogonality: A Simple View of What Makes BYOL Tickcited by 15
  5. 2021EigenGame Unloaded: When playing games is better than optimizingcited by 13
  6. 2022Differentially private Riemannian optimizationcited by 12
  7. 2023CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebracited by 7
  8. 2023Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Featurescited by 6
  9. 2021On the Second-order Convergence Properties of Random Search Methodscited by 5
  10. 2021FedPower: Privacy-Preserving Distributed Eigenspace Estimationcited by 4
  11. 2021Practical and Fast Momentum-Based Power Methodscited by 1
  12. 2022Universal Adversarial Directionscited by 1
6 more
  1. 2024Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reductioncited by 1
  2. 2025Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflationcited by 1
  3. 2021AgFlow: Fast Model Selection of Penalized PCA via Implicit Regularization Effects of Gradient Flowcited by 0
  4. 2022Stochastic Parallelizable Eigengap Dilation for Large Graph Clusteringcited by 0
  5. 2025k-SVD with Gradient Descentcited by 0
  6. 2026Incremental Aggregation on the Grassmannian for Asynchronous Eigenspace Computationcited by 0