mlsanityData to 14 Oct 2026

On the Escaping Efficiency of Distributed Adversarial Training Algorithms

arXiv 2509.11337 ↗First posted 14 Sep 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Adversarial training has been widely studied in recent years due to its role in improving model robustness against adversarial attacks. This paper focuses on comparing different distributed adversarial training algorithms--including centralized and decentralized strategies--within multi-agent learning environments. Previous studies have highlighted the importance of model flatness in determining robustness. To this end, we develop a general theoretical framework to study the escaping efficiency of these algorithms from local minima, which is closely related to the flatness of the resulting models. We show that when the perturbation bound is sufficiently small (i.e., when the attack strength is relatively mild) and a large batch size is used, decentralized adversarial training algorithms--including consensus and diffusion--are guaranteed to escape faster from local minima than the centralized strategy, thereby favoring flatter minima. However, as the perturbation bound increases, this trend may no longer hold. In the simulation results, we illustrate our theoretical findings and systematically compare the performance of models obtained through decentralized and centralized adversarial training algorithms. The results highlight the potential of decentralized strategies to enhance the robustness of models in distributed settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 14 Sep 2025.

Builds on

43 of 58 references.

References · 43

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  5. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  7. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  8. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  9. 2020Overfitting in adversarially robust deep learningcited by 506
  10. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  11. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 337
  12. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
31 more
  1. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
  2. 2021On the Convergence and Robustness of Adversarial Trainingcited by 130
  3. 2021Decentralized Federated Averagingcited by 114
  4. 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
  5. 2019Efficient Algorithms for Smooth Minimax Optimizationcited by 76
  6. 2020On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Themcited by 59
  7. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  8. 2021Consensus Control for Decentralized Deep Learningcited by 49
  9. 2019First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noisecited by 44
  10. 2021A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learningcited by 44
  11. 2022Topology-aware Generalization of Decentralized SGDcited by 38
  12. 2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
  13. 2019On the Influence of Bias-Correction on Distributed Stochastic Optimizationcited by 34
  14. 2021Power-law escape rate of SGDcited by 29
  15. 2022Beyond spectral gap: The role of the topology in decentralized learningcited by 27
  16. 2020SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothingcited by 26
  17. 2024On the Duality Between Sharpness-Aware Minimization and Adversarial Trainingcited by 24
  18. 2019Distributed Learning in Non-Convex Environments -- Part I: Agreement at a Linear Ratecited by 22
  19. 2021BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learningcited by 16
  20. 2023Decentralized SGD and Average-direction SAM are Asymptotically Equivalentcited by 15
  21. 2019Distributed Learning in Non-Convex Environments -- Part II: Polynomial Escape from Saddle-Pointscited by 13
  22. 2019Second-Order Guarantees of Stochastic Gradient Descent in Non-Convex Optimizationcited by 11
  23. 2022Networked Signal and Information Processingcited by 10
  24. 2023Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithmcited by 10
  25. 2013On the Learning Behavior of Adaptive Networks - Part II: Performance Analysiscited by 9
  26. 2022Distributed Adversarial Training to Robustify Deep Neural Networks at Scalecited by 9
  27. 2020Robust Deep Learning as Optimal Control: Insights and Convergence Guaranteescited by 7
  28. 2023Enforcing Privacy in Distributed Learning with Performance Guaranteescited by 3
  29. 2023Decentralized Adversarial Training over Graphscited by 3
  30. 2024On the Trade-off between Flatness and Optimization in Distributed Learningcited by 3
  31. 2025DICE: Data Influence Cascade in Decentralized Learningcited by 2

Built on by

Not cited yet.