# Personalized PCA: Decoupling Shared and Unique Features

arXiv: https://arxiv.org/abs/2207.08041 · HTML: https://mlsanity.com/paper/2207.08041/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 41 references that are in the map.
- First posted on arXiv 17 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Survey of Personalization Techniques for Federated Learning](https://mlsanity.com/paper/2003.08673/) (2020)
- [Turning Big data into tiny data: Constant-size coresets for k-means, PCA and projective clustering](https://mlsanity.com/paper/1807.04518/) (2018)
- [Guaranteed Matrix Completion via Non-convex Factorization](https://mlsanity.com/paper/1411.8003/) (2014)
- [Decentralized Riemannian Gradient Descent on the Stiefel Manifold](https://mlsanity.com/paper/2102.07091/) (2021)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [Fast and Simple PCA via Convex Optimization](https://mlsanity.com/paper/1509.05647/) (2015)
- [Exponentially convergent stochastic k-PCA without variance reduction](https://mlsanity.com/paper/1904.01750/) (2019)
- [Communication-efficient Algorithms for Distributed Stochastic Principal Component Analysis](https://mlsanity.com/paper/1702.08169/) (2017)
- [On the Local Linear Rate of Consensus on the Stiefel Manifold](https://mlsanity.com/paper/2101.09346/) (2021)
- [Distributed Principal Component Analysis with Limited Communication](https://mlsanity.com/paper/2110.14391/) (2021)
- [Distributed Estimation for Principal Component Analysis: an Enlarged Eigenspace Analysis](https://mlsanity.com/paper/2004.02336/) (2020)

## Built on by (most cited first, 17 of 17)

- [Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration](https://mlsanity.com/paper/2507.22170/) (2025)
- [Personalized Dictionary Learning for Heterogeneous Datasets](https://mlsanity.com/paper/2305.15311/) (2023)
- [Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective](https://mlsanity.com/paper/2411.07523/) (2024)
- [LEARNER: A Transfer Learning Method for Low-Rank Matrix Estimation](https://mlsanity.com/paper/2412.20605/) (2024)
- [Personalized Tucker Decomposition: Modeling Commonality and Peculiarity on Tensor Data](https://mlsanity.com/paper/2309.03439/) (2023)
- [A Generalized Mean Approach for Distributed-PCA](https://mlsanity.com/paper/2410.00397/) (2024)
- [Personalized Coupled Tensor Decomposition for Multimodal Data Fusion: Uniqueness and Algorithms](https://mlsanity.com/paper/2412.01102/) (2024)
- [Personalized Convolutional Dictionary Learning of Physiological Time Series](https://mlsanity.com/paper/2503.07687/) (2025)
- [Rethinking LoRA for Data Heterogeneous Federated Learning: Subspace and State Alignment](https://mlsanity.com/paper/2602.01746/) (2026)
- [ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning](https://mlsanity.com/paper/2402.12537/) (2024)
- [SpotDiff: Spotting and Disentangling Interference in Feature Space for Subject-Preserving Image Generation](https://mlsanity.com/paper/2510.07340/) (2025)
- [Joint estimation of high-dimensional spiked covariance matrices via a partially shared subspace](https://mlsanity.com/paper/2607.08123/) (2026)
- [Learning Driver Models for Automated Vehicles via Knowledge Sharing and Personalization](https://mlsanity.com/paper/2308.16870/) (2023)
- [eRPCA: Robust Principal Component Analysis for Exponential Family Distributions](https://mlsanity.com/paper/2310.19787/) (2023)
- [Client-Conditional Federated Learning via Local Training Data Statistics](https://mlsanity.com/paper/2603.11307/) (2026)
- [Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAS](https://mlsanity.com/paper/2607.18209/) (2026)
- [Personalized Federated Hierarchical Gaussian Processes for Privacy-Preserving Modeling of Heterogeneous Distributed Systems](https://mlsanity.com/paper/2609.19337/) (2026)

## Abstract (arXiv)

In this paper, we tackle a significant challenge in PCA: heterogeneity. When data are collected from different sources with heterogeneous trends while still sharing some congruency, it is critical to extract shared knowledge while retaining the unique features of each source. To this end, we propose personalized PCA (PerPCA), which uses mutually orthogonal global and local principal components to encode both unique and shared features. We show that, under mild conditions, both unique and shared features can be identified and recovered by a constrained optimization problem, even if the covariance matrices are immensely different. Also, we design a fully federated algorithm inspired by distributed Stiefel gradient descent to solve the problem. The algorithm introduces a new group of operations called generalized retractions to handle orthogonality constraints, and only requires global PCs to be shared across sources. We prove the linear convergence of the algorithm under suitable assumptions. Comprehensive numerical experiments highlight PerPCA's superior performance in feature extraction and prediction from heterogeneous datasets. As a systematic approach to decouple shared and unique features from heterogeneous datasets, PerPCA finds applications in several tasks, including video segmentation, topic extraction, and feature clustering.