mlsanityData to 7 Oct 2026

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

arXiv 2102.12920 ↗First posted 25 Feb 2021cs.LG cs.DCcited by 35

Not on the map. Why?

Abstract (arXiv)

Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation. As a flexible learning setting, federated learning has the potential to integrate with other learning frameworks. We conduct a focused survey of federated learning in conjunction with other learning algorithms. Specifically, we explore various learning algorithms to improve the vanilla federated averaging algorithm and review model fusion methods such as adaptive aggregation, regularization, clustered methods, and Bayesian methods. Following the emerging trends, we also discuss federated learning in the intersection with other learning paradigms, termed federated X learning, where X includes multitask learning, meta-learning, transfer learning, unsupervised learning, and reinforcement learning. In addition to reviewing state-of-the-art studies, this paper also identifies key challenges and applications in this field, while also highlighting promising future directions.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 75 of its 211 references that are in the map. First posted on arXiv 25 Feb 2021.

Builds on

75 of 211 references.

References · 75

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2019Advances and Open Problems in Federated Learningcited by 3,021
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  11. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  12. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
38 more
  1. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  2. 2020Adaptive Federated Optimizationcited by 1,036
  3. 2018How To Backdoor Federated Learningcited by 745
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  5. 2017Federated Multi-Task Learningcited by 669
  6. 2020Federated Learning with Matched Averagingcited by 564
  7. 2019Federated Learning with Personalization Layerscited by 533
  8. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  9. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  11. 2019Agnostic Federated Learningcited by 451
  12. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  13. 2019Fair Resource Allocation in Federated Learningcited by 389
  14. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  15. 2020Adaptive Personalized Federated Learningcited by 353
  16. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  17. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  18. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  19. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
  20. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
  21. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  22. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  23. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  24. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  25. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  26. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  27. 2019Model Fusion via Optimal Transportcited by 217
  28. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  29. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  30. 2020Threats to Federated Learning: A Surveycited by 171
  31. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  32. 2019Federated Evaluation of On-device Personalizationcited by 147
  33. 2019Federated Adversarial Domain Adaptationcited by 140
  34. 2019Federated Learning for Wireless Communications: Motivation, Opportunities and Challengescited by 128
  35. 2020Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learningcited by 112
  36. 2019Secure Federated Matrix Factorizationcited by 102
  37. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 99
  38. 2020Collaborative Fairness in Federated Learningcited by 95

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 5; 2024: 9; 2025: 10; 2026: 2.

35 papers.

Sort:

By area:

  1. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  2. 2022Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directionscited by 34
  3. 2023Federated Domain Generalization: A Surveycited by 25
  4. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  5. 2022Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phasescited by 18
  6. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  7. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  8. 2024Robust Training of Federated Models with Extremely Label Deficiencycited by 6
  9. 2023CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with Clustered Aggregation and Knowledge DIStilled Regularizationcited by 5
  10. 2023Rethinking Semi-Supervised Federated Learning: How to co-train fully-labeled and fully-unlabeled client imaging datacited by 4
  11. 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learningcited by 4
  12. 2022Bayesian data fusion with shared priorscited by 3
23 more
  1. 2023Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectivescited by 3
  2. 2021Gradual Federated Learning with Simulated Annealingcited by 2
  3. 2024Recommender Engine Driven Client Selection in Federated Brain Tumor Segmentationcited by 2
  4. 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applicationscited by 2
  5. 2025GC-Fed: Gradient Centralized Federated Learning with Partial Client Participationcited by 2
  6. 2023Federated Learning with Classifier Shift for Class Imbalancecited by 1
  7. 2024Federated Learning in Healthcare: Model Misconducts, Security, Challenges, Applications, and Future Research Directions -- A Systematic Reviewcited by 1
  8. 2025Federated Timeline Synthesis: Scalable and Private Methodology For Model Training and Deploymentcited by 1
  9. 2025Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insightscited by 1
  10. 2022TorchFL: A Performant Library for Bootstrapping Federated Learning Experimentscited by 0
  11. 2022GTFLAT: Game Theory Based Add-On For Empowering Federated Learning Aggregation Techniquescited by 0
  12. 2024Enhancing IoT Security Against DDoS Attacks through Federated Learningcited by 0
  13. 2024Heterogeneous Federated Learning with Convolutional and Spiking Neural Networkscited by 0
  14. 2024Federated Clustering: An Unsupervised Cluster-Wise Training for Decentralized Data Distributionscited by 0
  15. 2024FedBrain-Distill: Communication-Efficient Federated Brain Tumor Classification Using Ensemble Knowledge Distillation on Non-IID Datacited by 0
  16. 2024GAI-Enabled Explainable Personalized Federated Semi-Supervised Learningcited by 0
  17. 2025FL-CLEANER: byzantine and backdoor defense by CLustering Errors of Activation maps in Non-iid fedErated leaRningcited by 0
  18. 2025Task-Agnostic Federation over Decentralized Data: Research Landscape and Visionscited by 0
  19. 2025Traversal Learning: A Lossless And Efficient Distributed Learning Frameworkcited by 0
  20. 2025Progressive Size-Adaptive Federated Learning: A Comprehensive Framework for Heterogeneous Multi-Modal Data Systemscited by 0
  21. 2025From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0cited by 0
  22. 2026Beyond Weights and Gradients: A Taxonomy of Federated Learning Messagescited by 0
  23. 2026Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deploymentcited by 0