mlsanityData to 7 Oct 2026

Combating Exacerbated Heterogeneity for Robust Models in Federated Learning

arXiv 2303.00250 ↗First posted 1 Mar 2023cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Privacy and security concerns in real-world applications have led to the development of adversarially robust federated models. However, the straightforward combination between adversarial training and federated learning in one framework can lead to the undesired robustness deterioration. We discover that the attribution behind this phenomenon is that the generated adversarial data could exacerbate the data heterogeneity among local clients, making the wrapped federated learning perform poorly. To deal with this problem, we propose a novel framework called Slack Federated Adversarial Training (SFAT), assigning the client-wise slack during aggregation to combat the intensified heterogeneity. Theoretically, we analyze the convergence of the proposed method to properly relax the objective when combining federated learning and adversarial training. Experimentally, we verify the rationality and effectiveness of SFAT on various benchmarked and real-world datasets with different adversarial training and federated optimization methods. The code is publicly available at https://github.com/ZFancy/SFAT.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 51 of its 66 references that are in the map. First posted on arXiv 1 Mar 2023.

Builds on

51 of 66 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  11. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  12. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
38 more
  1. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  2. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  3. 2018Federated Learning with Non-IID Datacited by 1,099
  4. 2016Adversarial Machine Learning at Scalecited by 934
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  6. 2018LEAF: A Benchmark for Federated Settingscited by 780
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2013Network In Networkcited by 600
  11. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  12. 2019Agnostic Federated Learningcited by 451
  13. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  14. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  15. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  16. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  17. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  18. 2018Spatially Transformed Adversarial Examplescited by 195
  19. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  20. 2019Are Labels Required for Improving Adversarial Robustness?cited by 162
  21. 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 152
  22. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  23. 2019Adversarially Robust Distillationcited by 124
  24. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  25. 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 94
  26. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  27. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 55
  28. 2020Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Freecited by 48
  29. 2022Sparsity Winning Twice: Better Robust Generalization from More Efficient Trainingcited by 41
  30. 2020How benign is benign overfitting?cited by 40
  31. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  32. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  33. 2016Crowdsourcing in Computer Visioncited by 31
  34. 2020FAT: Federated Adversarial Trainingcited by 23
  35. 2021Adversarial training in communication constrained federated learningcited by 17
  36. 2021Neural Architecture Dilation for Adversarial Robustnesscited by 13
  37. 2021Certified Federated Adversarial Trainingcited by 6
  38. 2022Federated Learning Aggregation: New Robust Algorithms with Guaranteescited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3.

4 papers.

  1. 2023On Strengthening and Defending Graph Reconstruction Attack with Markov Chain Approximationcited by 16
  2. 2024Robust Training of Federated Models with Extremely Label Deficiencycited by 6
  3. 2024An Architecture Built for Federated Learning: Addressing Data Heterogeneity through Adaptive Normalization-Free Feature Recalibrationcited by 0
  4. 2024How Does the Smoothness Approximation Method Facilitate Generalization for Federated Adversarial Learning?cited by 0