mlsanityData to 7 Oct 2026

Personalized PCA: Decoupling Shared and Unique Features

arXiv 2207.08041 ↗First posted 17 Jul 2022cs.LG math.ST stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

In this paper, we tackle a significant challenge in PCA: heterogeneity. When data are collected from different sources with heterogeneous trends while still sharing some congruency, it is critical to extract shared knowledge while retaining the unique features of each source. To this end, we propose personalized PCA (PerPCA), which uses mutually orthogonal global and local principal components to encode both unique and shared features. We show that, under mild conditions, both unique and shared features can be identified and recovered by a constrained optimization problem, even if the covariance matrices are immensely different. Also, we design a fully federated algorithm inspired by distributed Stiefel gradient descent to solve the problem. The algorithm introduces a new group of operations called generalized retractions to handle orthogonality constraints, and only requires global PCs to be shared across sources. We prove the linear convergence of the algorithm under suitable assumptions. Comprehensive numerical experiments highlight PerPCA's superior performance in feature extraction and prediction from heterogeneous datasets. As a systematic approach to decouple shared and unique features from heterogeneous datasets, PerPCA finds applications in several tasks, including video segmentation, topic extraction, and feature clustering.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 13 of its 41 references that are in the map. First posted on arXiv 17 Jul 2022.

Builds on

13 of 41 references.

References · 13

  1. 2018LEAF: A Benchmark for Federated Settingscited by 780
  2. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  3. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  4. 2018Turning Big data into tiny data: Constant-size coresets for k-means, PCA and projective clusteringcited by 94
  5. 2014Guaranteed Matrix Completion via Non-convex Factorizationcited by 81
  6. 2021Decentralized Riemannian Gradient Descent on the Stiefel Manifoldcited by 33
  7. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  8. 2015Fast and Simple PCA via Convex Optimizationcited by 16
  9. 2019Exponentially convergent stochastic k-PCA without variance reductioncited by 12
  10. 2017Communication-efficient Algorithms for Distributed Stochastic Principal Component Analysiscited by 10
  11. 2021On the Local Linear Rate of Consensus on the Stiefel Manifoldcited by 8
  12. 2021Distributed Principal Component Analysis with Limited Communicationcited by 6
1 more
  1. 2020Distributed Estimation for Principal Component Analysis: an Enlarged Eigenspace Analysiscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 3; 2026: 5.

17 papers.

Sort:

By area:

  1. 2023Personalized Dictionary Learning for Heterogeneous Datasetscited by 5
  2. 2024Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspectivecited by 5
  3. 2025Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integrationcited by 4
  4. 2024LEARNER: A Transfer Learning Method for Low-Rank Matrix Estimationcited by 3
  5. 2023Personalized Tucker Decomposition: Modeling Commonality and Peculiarity on Tensor Datacited by 2
  6. 2024A Generalized Mean Approach for Distributed-PCAcited by 2
  7. 2024Personalized Coupled Tensor Decomposition for Multimodal Data Fusion: Uniqueness and Algorithmscited by 2
  8. 2025Personalized Convolutional Dictionary Learning of Physiological Time Seriescited by 2
  9. 2026Rethinking LoRA for Data Heterogeneous Federated Learning: Subspace and State Alignmentcited by 2
  10. 2024ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learningcited by 1
  11. 2025SpotDiff: Spotting and Disentangling Interference in Feature Space for Subject-Preserving Image Generationcited by 1
  12. 2023Learning Driver Models for Automated Vehicles via Knowledge Sharing and Personalizationcited by 0
5 more
  1. 2023eRPCA: Robust Principal Component Analysis for Exponential Family Distributionscited by 0
  2. 2026Client-Conditional Federated Learning via Local Training Data Statisticscited by 0
  3. 2026Joint estimation of high-dimensional spiked covariance matrices via a partially shared subspacecited by 0
  4. 2026Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAScited by 0
  5. 2026Personalized Federated Hierarchical Gaussian Processes for Privacy-Preserving Modeling of Heterogeneous Distributed Systemscited by 0