mlsanityData to 7 Oct 2026

Why adversarial training can hurt robust accuracy

arXiv 2203.02006 ↗First posted 3 Mar 2022cs.LG cs.CR cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Machine learning classifiers with high test accuracy often perform poorly under adversarial attacks. It is commonly believed that adversarial training alleviates this issue. In this paper, we demonstrate that, surprisingly, the opposite may be true -- Even though adversarial training helps when enough data is available, it may hurt robust generalization in the small sample size regime. We first prove this phenomenon for a high-dimensional linear classification setting with noiseless observations. Our proof provides explanatory insights that may also transfer to feature learning models. Further, we observe in experiments on standard image datasets that the same behavior occurs for perceptible attacks that effectively reduce class information such as mask attacks and object corruptions.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 49 of its 68 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

49 of 68 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  6. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  7. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  8. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  9. 2018Robustness May Be at Odds with Accuracycited by 873
  10. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  11. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  12. 2020Overfitting in adversarially robust deep learningcited by 504
37 more
  1. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  2. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  3. 2018Adversarially Robust Generalization Requires More Datacited by 318
  4. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  5. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  6. 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
  7. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  8. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  9. 2018Disentangling Adversarial Robustness and Generalizationcited by 171
  10. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  11. 2019Are Labels Required for Improving Adversarial Robustness?cited by 162
  12. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 148
  13. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  14. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  15. 2020Precise Tradeoffs in Adversarial Training for Linear Regressioncited by 76
  16. 2019Adversarially Robust Generalization Just Requires More Unlabeled Datacited by 75
  17. 2020Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalizationcited by 73
  18. 2019Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distancecited by 72
  19. 2019Defending Against Physically Realizable Attacks on Image Classificationcited by 66
  20. 2018Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Ratecited by 63
  21. 2020CAT: Customized Adversarial Training for Improved Robustnesscited by 61
  22. 2020On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Themcited by 59
  23. 2018Motivating the Rules of the Game for Adversarial Example Researchcited by 57
  24. 2019Instance adaptive adversarial training: Improved accuracy tradeoffs in neural netscited by 55
  25. 2019Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracycited by 49
  26. 2020Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attackscited by 45
  27. 2020How benign is benign overfitting?cited by 40
  28. 2020Provable tradeoffs in adversarially robust classificationcited by 39
  29. 2018Towards Imperceptible and Robust Adversarial Example Attacks against Neural Networkscited by 38
  30. 2013Margins, Shrinkage, and Boostingcited by 35
  31. 2020Towards Understanding Fast Adversarial Trainingcited by 32
  32. 2020More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Modelscited by 27
  33. 2019Transfer of Adversarial Robustness Between Perturbation Typescited by 25
  34. 2020Breaking certified defenses: Semantic adversarial examples with spoofed robustness certificatescited by 17
  35. 2021Interpolation can hurt robust generalization even when there is no noisecited by 15
  36. 2021Adversarial Perturbations Are Not So Weird: Entanglement of Robust and Non-Robust Features in Neural Network Classifierscited by 7
  37. 2021Data Quality Matters For Adversarial Training: An Empirical Studycited by 6

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 5; 2025: 1.

12 papers.

  1. 2023Devil is in Channels: Contrastive Single Domain Generalization for Medical Image Segmentationcited by 30
  2. 2023Improving the Robustness of Transformer-based Large Language Models with Dynamic Attentioncited by 10
  3. 2023Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approachcited by 8
  4. 2024Adversarially Robust Feature Learning for Breast Cancer Diagnosiscited by 3
  5. 2023Mitigating Adversarial Attacks in Federated Learning with Trusted Execution Environmentscited by 2
  6. 2024Specification Overfitting in Artificial Intelligencecited by 2
  7. 2023How robust accuracy suffers from certified training with convex relaxationscited by 1
  8. 2023Pelta: Shielding Transformers to Mitigate Evasion Attacks in Federated Learningcited by 1
  9. 2025On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classificationcited by 1
  10. 2024Accuracy of TextFooler black box adversarial attacks on 01 loss sign activation neural network ensemblecited by 0
  11. 2024Approximate Nullspace Augmented Finetuning for Robust Vision Transformerscited by 0
  12. 2024Top-GAP: Integrating Size Priors in CNNs for more Interpretability, Robustness, and Bias Mitigationcited by 0