# Improved Distributed Principal Component Analysis

arXiv: https://arxiv.org/abs/1408.5823 · HTML: https://mlsanity.com/paper/1408.5823/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2014.

## Built on by (most cited first, 14 of 14)

- [Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis](https://mlsanity.com/paper/2104.10095/) (2021)
- [EigenGame Unloaded: When playing games is better than optimizing](https://mlsanity.com/paper/2102.04152/) (2021)
- [Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning](https://mlsanity.com/paper/2309.05505/) (2023)
- [DeMuon: A Decentralized Muon for Matrix Optimization over Graphs](https://mlsanity.com/paper/2510.01377/) (2025)
- [Discrete Speech Unit Extraction via Independent Component Analysis](https://mlsanity.com/paper/2501.06562/) (2025)
- [Federated singular value decomposition for high dimensional data](https://mlsanity.com/paper/2205.12109/) (2022)
- [Communication-efficient distributed eigenspace estimation with arbitrary node failures](https://mlsanity.com/paper/2206.00127/) (2022)
- [Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis](https://mlsanity.com/paper/2303.17885/) (2023)
- [Distributed Differentially Private Data Analytics via Secure Sketching](https://mlsanity.com/paper/2412.00497/) (2024)
- [Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflation](https://mlsanity.com/paper/2502.17615/) (2025)
- [Collaborative Compressors in Distributed Mean Estimation with Limited Communication Budget](https://mlsanity.com/paper/2601.18950/) (2026)
- [Communication-efficient k-Means for Edge-based Machine Learning](https://mlsanity.com/paper/2102.04282/) (2021)
- [Distributed Clustering based on Distributional Kernel](https://mlsanity.com/paper/2409.09418/) (2024)
- [Information-Theoretic Bounds for Sparse Covariance Estimation in the Vertical-Split Distributed Model](https://mlsanity.com/paper/2606.07124/) (2026)

## Abstract (arXiv)

We study the distributed computing setting in which there are multiple servers, each holding a set of points, who wish to compute functions on the union of their point sets. A key task in this setting is Principal Component Analysis (PCA), in which the servers would like to compute a low dimensional subspace capturing as much of the variance of the union of their point sets as possible. Given a procedure for approximate PCA, one can use it to approximately solve ℓ2-error fitting problems such as k-means clustering and subspace clustering. The essential properties of an approximate distributed PCA algorithm are its communication cost and computational efficiency for a given desired accuracy in downstream applications. We give new algorithms and analyses for distributed PCA which lead to improved communication and computational costs for k-means clustering and related problems. Our empirical study on real world data shows a speedup of orders of magnitude, preserving communication with only a negligible degradation in solution quality. Some of these techniques we develop, such as a general transformation from a constant success probability subspace embedding to a high success probability subspace embedding with a dimension and sparsity independent of the success probability, may be of independent interest.