mlsanityData to 7 Oct 2026

From Distributed Machine Learning to Federated Learning: A Survey

arXiv 2104.14362 ↗First posted 29 Apr 2021cs.DC cs.AI cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

In recent years, data and computing resources are typically distributed in the devices of end users, various regions or organizations. Because of laws or regulations, the distributed data and computing resources cannot be directly shared among different regions or organizations for machine learning tasks. Federated learning emerges as an efficient approach to exploit distributed data and computing resources, so as to collaboratively train machine learning models, while obeying the laws and regulations and ensuring data security and data privacy. In this paper, we provide a comprehensive survey of existing works for federated learning. We propose a functional architecture of federated learning systems and a taxonomy of related techniques. Furthermore, we present the distributed training, data communication, and security of FL systems. Finally, we analyze their limitations and propose future research directions.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 106 of its 208 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

106 of 208 references.

References · 106

  1. 2017Graph Attention Networkscited by 8,854
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2016Deep Learning with Differential Privacycited by 2,682
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  9. 2019Towards Federated Learning at Scale: System Designcited by 826
  10. 2018LEAF: A Benchmark for Federated Settingscited by 780
  11. 2018How To Backdoor Federated Learningcited by 745
  12. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
38 more
  1. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  2. 2020Federated Learning with Matched Averagingcited by 564
  3. 2019Federated Learning with Personalization Layerscited by 533
  4. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  5. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  6. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  7. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  8. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  9. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  10. 2019Agnostic Federated Learningcited by 451
  11. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  12. 2020Backdoor Learning: A Surveycited by 412
  13. 2019Federated Machine Learning: Concept and Applicationscited by 409
  14. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  15. 2019Fair Resource Allocation in Federated Learningcited by 389
  16. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  17. 2018Sparsified SGD with Memorycited by 373
  18. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  19. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  20. 2020Adaptive Personalized Federated Learningcited by 353
  21. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  22. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  23. 2019Robust Aggregation for Federated Learningcited by 314
  24. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  25. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  26. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  27. 2019Can You Really Backdoor Federated Learning?cited by 250
  28. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
  29. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  30. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  31. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  32. 2019A Survey on Distributed Machine Learningcited by 187
  33. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  34. 2017Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryptioncited by 174
  35. 2020Threats to Federated Learning: A Surveycited by 171
  36. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  37. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  38. 2018Mitigating Sybils in Federated Learning Poisoningcited by 158

50 most cited of 106 All 106 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 10; 2023: 17; 2024: 17; 2025: 14; 2026: 4.

65 papers.

Sort:

By area:

  1. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  2. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  3. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  4. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  5. 2022No Free Lunch Theorem for Security and Utility in Federated Learningcited by 39
  6. 2021Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networkscited by 35
  7. 2023A Survey on Vertical Federated Learning: From a Layered Perspectivecited by 30
  8. 2022FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Servercited by 29
  9. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  10. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  11. 2022FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneitycited by 20
  12. 2022Multi-Job Intelligent Scheduling with Cross-Device Federated Learningcited by 20
38 more
  1. 2023Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Surveycited by 17
  2. 2024OpenFGL: A Comprehensive Benchmark for Federated Graph Learningcited by 16
  3. 2021HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environmentscited by 15
  4. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  5. 2023Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-artcited by 14
  6. 2024Trustworthy Federated Learning: Privacy, Security, and Beyondcited by 11
  7. 2023Efficient Asynchronous Federated Learning with Sparsification and Quantizationcited by 9
  8. 2024Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Surveycited by 9
  9. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  10. 2024Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learningcited by 8
  11. 2025Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustnesscited by 8
  12. 2024Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Datacited by 6
  13. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  14. 2022Federated Self-Supervised Learning for Acoustic Event Classificationcited by 5
  15. 2022Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resourcescited by 5
  16. 2023Federated Learning for 6G: Paradigms, Taxonomy, Recent Advances and Insightscited by 4
  17. 2024Decentralized Federated Learning with Model Caching on Mobile Agentscited by 4
  18. 2025SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignmentcited by 3
  19. 2025Deep Learning-based Techniques for Integrated Sensing and Communication Systems: State-of-the-Art, Challenges, and Opportunitiescited by 3
  20. 2021DeceFL: A Principled Decentralized Federated Learning Frameworkcited by 2
  21. 2022Decentralized digital twins of complex dynamical systemscited by 2
  22. 2023Federated K-Means Clustering via Dual Decomposition-based Distributed Optimizationcited by 2
  23. 2023On Mask-based Image Set Desensitization with Recognition Supportcited by 2
  24. 2024Resource Allocation and Workload Scheduling for Large-Scale Distributed Deep Learning: A Surveycited by 2
  25. 2025Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possiblecited by 2
  26. 2025Efficient Federated Learning with Timely Update Disseminationcited by 2
  27. 2025Efficient Federated Learning with Heterogeneous Data and Adaptive Dropoutcited by 2
  28. 2023Collaborative Distributed Machine Learningcited by 1
  29. 2024Gradient Coding in Decentralized Learning for Evading Stragglerscited by 1
  30. 2024Private Knowledge Sharing in Distributed Learning: A Surveycited by 1
  31. 2025Enhancing Air Quality Monitoring: A Brief Review of Federated Learning Advancescited by 1
  32. 2022FedVeca: Federated Vectorized Averaging on Non-IID Data with Adaptive Bi-directional Global Objectivecited by 0
  33. 2022Entry Dependent Expert Selection in Distributed Gaussian Processes Using Multilabel Classificationcited by 0
  34. 2023Hierarchical and Decentralised Federated Learningcited by 0
  35. 2023MetisFL: An Embarrassingly Parallelized Controller for Scalable & Efficient Federated Learning Workflowscited by 0
  36. 2023The Landscape of Modern Machine Learning: A Review of Machine, Distributed and Federated Learningcited by 0
  37. 2023Horizontal Federated Computer Visioncited by 0
  38. 2024Vertical Federated Image Segmentationcited by 0

50 most cited of 65 All 65 (JSON)