# Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark

arXiv: https://arxiv.org/abs/2311.06750 · HTML: https://mlsanity.com/paper/2311.06750/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- Builds on 171 of its 309 references that are in the map.
- First posted on arXiv 12 Nov 2023.

## Builds on (most cited first, 30 of 171)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)

## Built on by (most cited first, 30 of 99)

- [A Review of Graph Neural Networks in Epidemic Modeling](https://mlsanity.com/paper/2403.19852/) (2024)
- [Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversity](https://mlsanity.com/paper/2405.16585/) (2024)
- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [All in One Framework for Multimodal Re-identification in the Wild](https://mlsanity.com/paper/2405.04741/) (2024)
- [Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model](https://mlsanity.com/paper/2503.04543/) (2025)
- [FedA3I: Annotation Quality-Aware Aggregation for Federated Medical Image Segmentation against Heterogeneous Annotation Noise](https://mlsanity.com/paper/2312.12838/) (2023)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization](https://mlsanity.com/paper/2401.10272/) (2024)
- [Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning](https://mlsanity.com/paper/2503.06457/) (2025)
- [LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models](https://mlsanity.com/paper/2503.16843/) (2025)
- [Adaptive High-Frequency Transformer for Diverse Wildlife Re-Identification](https://mlsanity.com/paper/2410.06977/) (2024)
- [CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data](https://mlsanity.com/paper/2312.08648/) (2023)
- [SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead](https://mlsanity.com/paper/2406.00431/) (2024)
- [Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.19101/) (2025)
- [SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devices](https://mlsanity.com/paper/2505.00365/) (2025)
- [FedMLLM: Federated Fine-tuning MLLM on Multimodal Heterogeneity Data](https://mlsanity.com/paper/2411.14717/) (2024)
- [S2FGL: Spatial Spectral Federated Graph Learning](https://mlsanity.com/paper/2507.02409/) (2025)
- [From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matching](https://mlsanity.com/paper/2404.17805/) (2024)
- [FedMLP: Federated Multi-Label Medical Image Classification under Task Heterogeneity](https://mlsanity.com/paper/2406.18995/) (2024)
- [FBFL: A Field-Based Coordination Approach for Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2502.08577/) (2025)
- [Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients](https://mlsanity.com/paper/2503.09206/) (2025)
- [From Laboratory to Real World: A New Benchmark Towards Privacy-Preserved Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2503.12232/) (2025)
- [ProFed: a Benchmark for Proximity-based non-IID Federated Learning](https://mlsanity.com/paper/2503.20618/) (2025)
- [Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspective](https://mlsanity.com/paper/2502.13732/) (2025)
- [FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Data](https://mlsanity.com/paper/2504.11903/) (2025)
- [FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2510.20250/) (2025)
- [Federated Automatic Latent Variable Selection in Multi-output Gaussian Processes](https://mlsanity.com/paper/2407.16935/) (2024)
- [Communication-Efficient Federated Group Distributionally Robust Optimization](https://mlsanity.com/paper/2410.06369/) (2024)
- [FairFML: Fair Federated Machine Learning with a Case Study on Reducing Gender Disparities in Cardiac Arrest Outcome Prediction](https://mlsanity.com/paper/2410.17269/) (2024)
- [Masked Autoencoders are Parameter-Efficient Federated Continual Learners](https://mlsanity.com/paper/2411.01916/) (2024)

## Abstract (arXiv)

Federated learning has emerged as a promising paradigm for privacy-preserving collaboration among different parties. Recently, with the popularity of federated learning, an influx of approaches have delivered towards different realistic challenges. In this survey, we provide a systematic overview of the important and recent developments of research on federated learning. Firstly, we introduce the study history and terminology definition of this area. Then, we comprehensively review three basic lines of research: generalization, robustness, and fairness, by introducing their respective background concepts, task settings, and main challenges. We also offer a detailed overview of representative literature on both methods and datasets. We further benchmark the reviewed methods on several well-known datasets. Finally, we point out several open issues in this field and suggest opportunities for further research. We also provide a public website to continuously track developments in this fast advancing field: https://github.com/WenkeHuang/MarsFL.