mlsanityData to 14 Oct 2026

From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challenges

arXiv 2503.07505 ↗First posted 10 Mar 2025cs.LG cs.AI cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) enables collaborative learning without directly sharing individual's raw data. FL can be implemented in either a centralized (server-based) or decentralized (peer-to-peer) manner. In this survey, we present a novel perspective: the fundamental difference between centralized FL (CFL) and decentralized FL (DFL) is not merely the network topology, but the underlying training protocol: separate aggregation vs. joint optimization. We argue that this distinction in protocol leads to significant differences in model utility, privacy preservation, and robustness to attacks. We systematically review and categorize existing works in both CFL and DFL according to the type of protocol they employ. This taxonomy provides deeper insights into prior research and clarifies how various approaches relate or differ. Through our analysis, we identify key gaps in the literature. In particular, we observe a surprising lack of exploration of DFL approaches based on distributed optimization methods, despite their potential advantages. We highlight this under-explored direction and call for more research on leveraging distributed optimization for federated learning. Overall, this work offers a comprehensive overview from centralized to decentralized FL, sheds new light on the core distinctions between approaches, and outlines open challenges and future directions for the field.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 91 of its 153 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

91 of 153 references.

References · 91

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  6. 2016An overview of gradient descent optimization algorithmscited by 1,033
  7. 2019Deep Leakage from Gradientscited by 1,021
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  9. 2021Membership Inference Attacks From First Principlescited by 878
  10. 2021Model-Contrastive Federated Learningcited by 766
  11. 2018How To Backdoor Federated Learningcited by 765
  12. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
38 more
  1. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  2. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  3. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  4. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  5. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  6. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  7. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 493
  8. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
  9. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  10. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  11. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
  12. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
  13. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
  14. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  15. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  16. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 246
  17. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 199
  18. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  19. 2018D2: Decentralized Training over Decentralized Datacited by 140
  20. 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 132
  21. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 128
  22. 2020Backdoor Attacks to Graph Neural Networkscited by 128
  23. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 127
  24. 2019BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learningcited by 117
  25. 2021Decentralized Federated Averagingcited by 114
  26. 2019Bit-Flip Attack: Crushing Neural Network with Progressive Bit Searchcited by 108
  27. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  28. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
  29. 2021Gradient Inversion with Generative Image Priorcited by 101
  30. 2020Local and Central Differential Privacy for Robustness and Privacy in Federated Learningcited by 78
  31. 2019Diversity-Sensitive Conditional Generative Adversarial Networkscited by 77
  32. 2020Knowledge-Enriched Distributional Model Inversion Attackscited by 67
  33. 2019Decentralized Federated Learning: A Segmented Gossip Approachcited by 64
  34. 2019Federated Learning with Bayesian Differential Privacycited by 60
  35. 2020XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learningcited by 58
  36. 2013On the Linear Convergence of the ADMM in Decentralized Consensus Optimizationcited by 57
  37. 2024Byzantine-Robust Decentralized Federated Learningcited by 48
  38. 2021GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learningcited by 36

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Byzantine-Resilient Federated Learning via Distributed Optimizationcited by 3
  2. 2025Generalization Error Analysis for Attack-Free and Byzantine-Resilient Decentralized Learning with Data Heterogeneitycited by 1
  3. 2026Decision-Focused Federated Learning Under Heterogeneous Objectives and Constraintscited by 1
  4. 2025DFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learningcited by 0