mlsanityData to 7 Oct 2026

Balancing Similarity and Complementarity for Federated Learning

arXiv 2405.09892 ↗First posted 16 May 2024cs.LG cs.DCcited by 7

Not on the map. Why?

Abstract (arXiv)

In mobile and IoT systems, Federated Learning (FL) is increasingly important for effectively using data while maintaining user privacy. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data, arising from numerous clients and diverse data sources. This requires strategic cooperation, often with clients having similar characteristics. However, we are interested in a fundamental question: does achieving optimal cooperation necessarily entail cooperating with the most similar clients? Typically, significant model performance improvements are often realized not by partnering with the most similar models, but through leveraging complementary data. Our theoretical and empirical analyses suggest that optimal cooperation is achieved by enhancing complementarity in feature distribution while restricting the disparity in the correlation between features and targets. Accordingly, we introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation. Our framework aims to approximate an optimal cooperation network for each client by optimizing a weighted sum of model similarity and feature complementarity. The strength of textttFedSaC lies in its adaptability to various levels of data heterogeneity and multimodal scenarios. Our comprehensive unimodal and multimodal experiments demonstrate that textttFedSaC markedly surpasses other state-of-the-art FL methods.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 28 of its 47 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

28 of 47 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2021Model-Contrastive Federated Learningcited by 762
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  7. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  8. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  9. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  10. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  11. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  12. 2019Agnostic Federated Learningcited by 451
16 more
  1. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  2. 2020Adaptive Personalized Federated Learningcited by 353
  3. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  4. 2021Personalized Federated Learning using Hypernetworkscited by 210
  5. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  6. 2019Federated Adversarial Domain Adaptationcited by 140
  7. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  8. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 85
  9. 2021Personalized Federated Learning through Local Memorizationcited by 57
  10. 2022Personalized Subgraph Federated Learningcited by 54
  11. 2022Federated Learning under Distributed Concept Driftcited by 35
  12. 2021Multimodal Federated Learning on IoT Datacited by 32
  13. 2023AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Drivingcited by 32
  14. 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
  15. 2021Collaboration Equilibrium in Federated Learningcited by 15
  16. 2021FRuDA: Framework for Distributed Adversarial Domain Adaptationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Datacited by 7
  2. 2025Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspectivecited by 4
  3. 2025AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Datacited by 3
  4. 2025Adaptive Prototype Knowledge Transfer for Federated Learning with Mixed Modalities and Heterogeneous Taskscited by 0
  5. 2025Clustered Federated Learning via Embedding Distributionscited by 0
  6. 2026Federated Personalization of Early-Exit Networkscited by 0
  7. 2026FedDRAW: Federated Dual Reputation Annealing Weighting for Heterogeneous Multi-Institutional Chest Radiograph Classificationcited by 0