# Federated Mutual Learning

arXiv: https://arxiv.org/abs/2006.16765 · HTML: https://mlsanity.com/paper/2006.16765/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2020.

## Built on by (most cited first, 30 of 71)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Federated Learning with Label Distribution Skew via Logits Calibration](https://mlsanity.com/paper/2209.00189/) (2022)
- [FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models](https://mlsanity.com/paper/2310.10049/) (2023)
- [Decentralized Federated Learning through Proxy Model Sharing](https://mlsanity.com/paper/2111.11343/) (2021)
- [FedGH: Heterogeneous Federated Learning with Generalized Global Header](https://mlsanity.com/paper/2303.13137/) (2023)
- [FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning](https://mlsanity.com/paper/2401.03230/) (2024)
- [Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2307.09182/) (2023)
- [Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning](https://mlsanity.com/paper/2309.16286/) (2023)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning](https://mlsanity.com/paper/2107.13892/) (2021)
- [Personalized Retrogress-Resilient Framework for Real-World Medical Federated Learning](https://mlsanity.com/paper/2110.00394/) (2021)
- [Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning](https://mlsanity.com/paper/2202.05318/) (2022)
- [Federated Model Heterogeneous Matryoshka Representation Learning](https://mlsanity.com/paper/2406.00488/) (2024)
- [Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matching](https://mlsanity.com/paper/2407.05005/) (2024)
- [Peer-to-Peer Federated Continual Learning for Naturalistic Driving Action Recognition](https://mlsanity.com/paper/2304.07421/) (2023)
- [FedH2L: Federated Learning with Model and Statistical Heterogeneity](https://mlsanity.com/paper/2101.11296/) (2021)
- [PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmark](https://mlsanity.com/paper/2312.04992/) (2023)
- [An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learning](https://mlsanity.com/paper/2403.15760/) (2024)
- [Universal Domain Adaptation via Compressive Attention Matching](https://mlsanity.com/paper/2304.11862/) (2023)
- [FedHe: Heterogeneous Models and Communication-Efficient Federated Learning](https://mlsanity.com/paper/2110.09910/) (2021)
- [FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks](https://mlsanity.com/paper/2201.03169/) (2022)
- [Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations](https://mlsanity.com/paper/2405.09839/) (2024)
- [Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures](https://mlsanity.com/paper/2411.19128/) (2024)
- [InOR-Net: Incremental 3D Object Recognition Network for Point Cloud Representation](https://mlsanity.com/paper/2302.09886/) (2023)
- [MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning](https://mlsanity.com/paper/2601.22416/) (2026)
- [FedMe: Federated Learning via Model Exchange](https://mlsanity.com/paper/2110.07868/) (2021)
- [Find Your Friends: Personalized Federated Learning with the Right Collaborators](https://mlsanity.com/paper/2210.06597/) (2022)
- [FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning](https://mlsanity.com/paper/2312.09006/) (2023)
- [pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharing](https://mlsanity.com/paper/2311.06879/) (2023)
- [FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning](https://mlsanity.com/paper/2501.05496/) (2025)

## Abstract (arXiv)

Federated learning (FL) enables collaboratively training deep learning models on decentralized data. However, there are three types of heterogeneities in FL setting bringing about distinctive challenges to the canonical federated learning algorithm (FedAvg). First, due to the Non-IIDness of data, the global shared model may perform worse than local models that solely trained on their private data; Second, the objective of center server and clients may be different, where center server seeks for a generalized model whereas client pursue a personalized model, and clients may run different tasks; Third, clients may need to design their customized model for various scenes and tasks; In this work, we present a novel federated learning paradigm, named Federated Mutual Leaning (FML), dealing with the three heterogeneities. FML allows clients training a generalized model collaboratively and a personalized model independently, and designing their private customized models. Thus, the Non-IIDness of data is no longer a bug but a feature that clients can be personally served better. The experiments show that FML can achieve better performance than alternatives in typical FL setting, and clients can be benefited from FML with different models and tasks.