# Balancing Similarity and Complementarity for Federated Learning

arXiv: https://arxiv.org/abs/2405.09892 · HTML: https://mlsanity.com/paper/2405.09892/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 16 May 2024.

## Builds on (most cited first, 28 of 28)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [Personalized Cross-Silo Federated Learning on Non-IID Data](https://mlsanity.com/paper/2007.03797/) (2020)
- [Personalized Federated Learning using Hypernetworks](https://mlsanity.com/paper/2103.04628/) (2021)
- [cpSGD: Communication-efficient and differentially-private distributed SGD](https://mlsanity.com/paper/1805.10559/) (2018)
- [Federated Adversarial Domain Adaptation](https://mlsanity.com/paper/1911.02054/) (2019)
- [Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2106.06047/) (2021)
- [Multimodal Federated Learning via Contrastive Representation Ensemble](https://mlsanity.com/paper/2302.08888/) (2023)
- [Personalized Federated Learning through Local Memorization](https://mlsanity.com/paper/2111.09360/) (2021)
- [Personalized Subgraph Federated Learning](https://mlsanity.com/paper/2206.10206/) (2022)
- [Federated Learning under Distributed Concept Drift](https://mlsanity.com/paper/2206.00799/) (2022)
- [Multimodal Federated Learning on IoT Data](https://mlsanity.com/paper/2109.04833/) (2021)
- [AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Driving](https://mlsanity.com/paper/2302.08646/) (2023)
- [Confidence-aware Personalized Federated Learning via Variational Expectation Maximization](https://mlsanity.com/paper/2305.12557/) (2023)
- [Collaboration Equilibrium in Federated Learning](https://mlsanity.com/paper/2108.07926/) (2021)
- [FRuDA: Framework for Distributed Adversarial Domain Adaptation](https://mlsanity.com/paper/2112.13381/) (2021)

## Built on by (most cited first, 7 of 7)

- [Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data](https://mlsanity.com/paper/2510.22880/) (2025)
- [Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspective](https://mlsanity.com/paper/2502.13732/) (2025)
- [AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data](https://mlsanity.com/paper/2505.12245/) (2025)
- [Adaptive Prototype Knowledge Transfer for Federated Learning with Mixed Modalities and Heterogeneous Tasks](https://mlsanity.com/paper/2502.04400/) (2025)
- [Clustered Federated Learning via Embedding Distributions](https://mlsanity.com/paper/2506.07769/) (2025)
- [Federated Personalization of Early-Exit Networks](https://mlsanity.com/paper/2601.10015/) (2026)
- [FedDRAW: Federated Dual Reputation Annealing Weighting for Heterogeneous Multi-Institutional Chest Radiograph Classification](https://mlsanity.com/paper/2609.05223/) (2026)

## Abstract (arXiv)

In mobile and IoT systems, Federated Learning (FL) is increasingly important for effectively using data while maintaining user privacy. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data, arising from numerous clients and diverse data sources. This requires strategic cooperation, often with clients having similar characteristics. However, we are interested in a fundamental question: does achieving optimal cooperation necessarily entail cooperating with the most similar clients? Typically, significant model performance improvements are often realized not by partnering with the most similar models, but through leveraging complementary data. Our theoretical and empirical analyses suggest that optimal cooperation is achieved by enhancing complementarity in feature distribution while restricting the disparity in the correlation between features and targets. Accordingly, we introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation. Our framework aims to approximate an optimal cooperation network for each client by optimizing a weighted sum of model similarity and feature complementarity. The strength of textttFedSaC lies in its adaptability to various levels of data heterogeneity and multimodal scenarios. Our comprehensive unimodal and multimodal experiments demonstrate that textttFedSaC markedly surpasses other state-of-the-art FL methods.