Data to 13 Oct 2026

Overfitting in adversarially robust deep learning

arXiv 2002.11569 ↗First posted 26 Feb 2020cs.LG stat.MLcited by 505

Not on the map. Why?

Abstract (arXiv)

It is common practice in deep learning to use overparameterized networks and train for as long as possible; there are numerous studies that show, both theoretically and empirically, that such practices surprisingly do not unduly harm the generalization performance of the classifier. In this paper, we empirically study this phenomenon in the setting of adversarially trained deep networks, which are trained to minimize the loss under worst-case adversarial perturbations. We find that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training across multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) and perturbation models (ℓ_∞ and ℓ2). Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting. Finally, we study several classical and modern deep learning remedies for overfitting, including regularization and data augmentation, and find that no approach in isolation improves significantly upon the gains achieved by early stopping. All code for reproducing the experiments as well as pretrained model weights and training logs can be found at https://github.com/locuslab/robustoverfitting.

Key facts. Cited by 505 papers in the map (data through 13 Oct 2026). First posted on arXiv 26 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 95; 2022: 129; 2023: 112; 2024: 82; 2025: 56; 2026: 31.

505 papers.

Sort:

By area:

2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 278
2021Improving Robustness using Generated Datacited by 234
2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 207
2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 203
2021Data Augmentation Can Improve Robustnesscited by 193
2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
2021A Universal Law of Robustness via Isoperimetrycited by 151
2022Towards Understanding Sharpness-Aware Minimizationcited by 146
2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 110

38 more

2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
2021Trustworthy AI: A Computational Perspectivecited by 93
2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 77
2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 74
2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
2022A Light Recipe to Train Robust Vision Transformerscited by 64
2022When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecturecited by 64
2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 59
2023Decoupled Kullback-Leibler Divergence Losscited by 58
2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
2021Exploring Memorization in Adversarial Trainingcited by 55
2021Reliable Adversarial Distillation with Unreliable Teacherscited by 55
2023CFA: Class-wise Calibrated Fair Adversarial Trainingcited by 53
2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 51
2021Adversarial Robustness under Long-Tailed Distributioncited by 50
2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 50
2021SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustnesscited by 49
2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
2022Towards Robust Referring Image Segmentationcited by 49
2024Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustnesscited by 48
2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Modelscited by 48
2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
2021Random Noise Defense Against Query-Based Black-Box Attackscited by 46
2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
2022A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologiescited by 44
2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
2023Data Augmentation Alone Can Improve Adversarial Trainingcited by 43
2023Rethinking Confidence Calibration for Failure Predictioncited by 43
2021Consistency Regularization for Adversarial Robustnesscited by 42
2022Make Some Noise: Reliable and Efficient Single-Step Adversarial Trainingcited by 42
2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
2021Subspace Adversarial Trainingcited by 41
2022Sparsity Winning Twice: Better Robust Generalization from More Efficient Trainingcited by 41
2021Boosting Fast Adversarial Training with Learnable Adversarial Initializationcited by 40

50 most cited of 505 All 505 (JSON)