mlsanityData to 14 Oct 2026

Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces

arXiv 2209.10526 ↗First posted 21 Sep 2022cs.LG cs.AI cs.DCcited by 36

Not on the map. Why?

Abstract (arXiv)

Clustered federated learning (FL) has been shown to produce promising results by grouping clients into clusters. This is especially effective in scenarios where separate groups of clients have significant differences in the distributions of their local data. Existing clustered FL algorithms are essentially trying to group together clients with similar distributions so that clients in the same cluster can leverage each other's data to better perform federated learning. However, prior clustered FL algorithms attempt to learn these distribution similarities indirectly during training, which can be quite time consuming as many rounds of federated learning may be required until the formation of clusters is stabilized. In this paper, we propose a new approach to federated learning that directly aims to efficiently identify distribution similarities among clients by analyzing the principal angles between the client data subspaces. Each client applies a truncated singular value decomposition (SVD) step on its local data in a single-shot manner to derive a small set of principal vectors, which provides a signature that succinctly captures the main characteristics of the underlying distribution. This small set of principal vectors is provided to the server so that the server can directly identify distribution similarities among the clients to form clusters. This is achieved by comparing the similarities of the principal angles between the client data subspaces spanned by those principal vectors. The approach provides a simple, yet effective clustered FL framework that addresses a broad range of data heterogeneity issues beyond simpler forms of Non-IIDness like label skews. Our clustered FL approach also enables convergence guarantees for non-convex objectives. Our code is available at https://github.com/MMorafah/PACFL.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 25 of its 50 references that are in the map. First posted on arXiv 21 Sep 2022.

Builds on

25 of 50 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  7. 2018Federated Learning with Non-IID Datacited by 1,108
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  9. 2017Federated Multi-Task Learningcited by 672
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  11. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  12. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
13 more
  1. 2020Adaptive Personalized Federated Learningcited by 358
  2. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  3. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  4. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  5. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 153
  6. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  7. 2015Network Lasso: Clustering and Optimization in Large Graphscited by 43
  8. 2021Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneitycited by 27
  9. 2022FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distributioncited by 22
  10. 2021Federated Learning with Taskonomy for Non-IID Datacited by 21
  11. 2020Communication-Efficient Distributed SVD via Local Power Iterationscited by 12
  12. 2022Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Datacited by 9
  13. 2020Federated Learning From Big Data Over Networkscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 10; 2025: 11; 2026: 8.

36 papers.

Sort:

By area:

  1. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
  2. 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 46
  3. 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 31
  4. 2022Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarkscited by 24
  5. 2024Personalized Federated Continual Learning via Multi-granularity Promptcited by 20
  6. 2023Optimizing the Collaboration Structure in Cross-Silo Federated Learningcited by 19
  7. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  8. 2024Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Datacited by 11
  9. 2022When Do Curricula Work in Federated Learning?cited by 9
  10. 2024FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clusteringcited by 9
  11. 2024FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clusteringcited by 9
  12. 2023A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Designcited by 5
24 more
  1. 2022Clustered Federated Learning based on Nonconvex Pairwise Fusioncited by 4
  2. 2023Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologiescited by 3
  3. 2025A Survey on Foundation Models for Personalized Federated Intelligencecited by 3
  4. 2024Geometrically Inspired Kernel Machines for Collaborative Learning Beyond Gradient Descentcited by 2
  5. 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applicationscited by 2
  6. 2025Operator-Theoretic Framework for Gradient-Free Federated Learningcited by 2
  7. 2026Few-for-Many Personalized Federated Learningcited by 2
  8. 2024FedRAV: Hierarchically Federated Region-Learning for Traffic Object Classification of Autonomous Vehiclescited by 1
  9. 2025MMiC: Mitigating Modality Incompleteness in Clustered Federated Learningcited by 1
  10. 2025Distilling A Universal Expert from Clustered Federated Learningcited by 1
  11. 2025CLoVE: Personalized Federated Learning through Clustering of Loss Vector Embeddingscited by 1
  12. 2024DA-PFL: Dynamic Affinity Aggregation for Personalized Federated Learningcited by 0
  13. 2024Equitable Federated Learning with Activation Clusteringcited by 0
  14. 2025Stratify: Rethinking Federated Learning for Non-IID Data through Balanced Samplingcited by 0
  15. 2025Tackling Heterogeneity in Federated Learning via Variance-Reduced Boltzmann Sampling within Homogeneous Social Coalitionscited by 0
  16. 2025Clustered Federated Learning via Embedding Distributionscited by 0
  17. 2025EBS-CFL: Efficient and Byzantine-robust Secure Clustered Federated Learningcited by 0
  18. 2026FedBiCross: Personalized One-Shot Federated Learning on Medical Imagescited by 0
  19. 2026FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environmentscited by 0
  20. 2026Differentially Private Clustered Federated Learning with Privacy-Preserving Initialization and Normality-Driven Aggregationcited by 0
  21. 2026FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learningcited by 0
  22. 2026Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadatacited by 0
  23. 2026FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneitycited by 0
  24. 2026CF-LoRA: Decoupled Factor Aggregation and Adaptation-Aware Client Clustering for Federated LoRA Fine-Tuningcited by 0