mlsanityData to 7 Oct 2026

A survey on secure decentralized optimization and learning

arXiv 2408.08628 ↗First posted 16 Aug 2024cs.LG math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

Decentralized optimization has become a standard paradigm for solving large-scale decision-making problems and training large machine learning models without centralizing data. However, this paradigm introduces new privacy and security risks, with malicious agents potentially able to infer private data or impair the model accuracy. Over the past decade, significant advancements have been made in developing secure decentralized optimization and learning frameworks and algorithms. This survey provides a comprehensive tutorial on these advancements. We begin with the fundamentals of decentralized optimization and learning, highlighting centralized aggregation and distributed consensus as key modules exposed to security risks in federated and distributed optimization, respectively. Next, we focus on privacy-preserving algorithms, detailing three cryptographic tools and their integration into decentralized optimization and learning systems. Additionally, we examine resilient algorithms, exploring the design and analysis of resilient aggregation and consensus protocols that support these systems. We conclude the survey by discussing current trends and potential future directions.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 128 of its 256 references that are in the map. First posted on arXiv 16 Aug 2024.

Builds on

128 of 256 references.

References · 128

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2016Adversarial Machine Learning at Scalecited by 934
  8. 2018How To Backdoor Federated Learningcited by 745
  9. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  10. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  11. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  12. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
38 more
  1. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  2. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  3. 2019Robust Aggregation for Federated Learningcited by 314
  4. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  5. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  6. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  7. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  8. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 125
  9. 2017Network Topology and Communication-Computation Tradeoffs in Decentralized Optimizationcited by 119
  10. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  11. 2018Byzantine Stochastic Gradient Descentcited by 81
  12. 2015Massively Parallel Methods for Deep Reinforcement Learningcited by 80
  13. 2019Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attackscited by 79
  14. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  15. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 64
  16. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  17. 2020FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learningcited by 60
  18. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  19. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 56
  20. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 56
  21. 2019Free-riders in Federated Learning: Attacks and Defensescited by 48
  22. 2017Sub-Gaussian estimators of the mean of a random vectorcited by 47
  23. 2019BRIDGE: Byzantine-resilient Decentralized Gradient Descentcited by 45
  24. 2017ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learningcited by 36
  25. 2022Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directionscited by 34
  26. 2022Joint Privacy Enhancement and Quantization in Federated Learningcited by 34
  27. 2018Data-Enabled Predictive Control: In the Shallows of the DeePCcited by 33
  28. 2022Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guaranteescited by 30
  29. 2020Byzantine-resilient Decentralized Stochastic Gradient Descentcited by 28
  30. 2020A general framework for decentralized optimization with first-order methodscited by 28
  31. 2021Approximate Byzantine Fault-Tolerance in Distributed Optimizationcited by 27
  32. 2022Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averagingcited by 27
  33. 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneitycited by 25
  34. 2022Towards Efficient Communications in Federated Learning: A Contemporary Surveycited by 24
  35. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  36. 2014Differentially Private Distributed Optimizationcited by 23
  37. 2022Byzantine-Resilient Decentralized Stochastic Optimization with Robust Aggregation Rulescited by 23
  38. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22

50 most cited of 128 All 128 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filteringcited by 1
  2. 2026Communication-Efficient Distributed Learning with Differential Privacycited by 1
  3. 2025Personalized and Resilient Distributed Learning Through Opinion Dynamicscited by 0
  4. 2026Topology-Independent Robustness of the Weighted Mean under Label Poisoning Attacks in Heterogeneous Decentralized Learningcited by 0