mlsanityData to 7 Oct 2026

Understanding Robust Overfitting of Adversarial Training and Beyond

arXiv 2206.08675 ↗First posted 17 Jun 2022cs.LG cs.AIcited by 50

Not on the map. Why?

Abstract (arXiv)

Robust overfitting widely exists in adversarial training of deep networks. The exact underlying reasons for this are still not completely understood. Here, we explore the causes of robust overfitting by comparing the data distribution of non-overfit (weak adversary) and overfitted (strong adversary) adversarial training, and observe that the distribution of the adversarial data generated by weak adversary mainly contain small-loss data. However, the adversarial data generated by strong adversary is more diversely distributed on the large-loss data and the small-loss data. Given these observations, we further designed data ablation adversarial training and identify that some small-loss data which are not worthy of the adversary strength cause robust overfitting in the strong adversary mode. To relieve this issue, we propose minimum loss constrained adversarial training (MLCAT): in a minibatch, we learn large-loss data as usual, and adopt additional measures to increase the loss of the small-loss data. Technically, MLCAT hinders data fitting when they become easy to learn to prevent robust overfitting; philosophically, MLCAT reflects the spirit of turning waste into treasure and making the best use of each adversarial data; algorithmically, we designed two realizations of MLCAT, and extensive experiments demonstrate that MLCAT can eliminate robust overfitting and further boost adversarial robustness.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 35 of its 42 references that are in the map. First posted on arXiv 17 Jun 2022.

Builds on

35 of 42 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Wide Residual Networkscited by 3,057
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  7. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  10. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  11. 2020Overfitting in adversarially robust deep learningcited by 504
  12. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
23 more
  1. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  2. 2018Adversarially Robust Generalization Requires More Datacited by 318
  3. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  4. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  5. 2018Adversarial Logit Pairingcited by 224
  6. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  7. 2020Bag of Tricks for Adversarial Trainingcited by 199
  8. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  9. 2019Are Labels Required for Improving Adversarial Robustness?cited by 162
  10. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  11. 2018Curriculum Adversarial Trainingcited by 89
  12. 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
  13. 2019Adversarially Robust Generalization Just Requires More Unlabeled Datacited by 75
  14. 2020Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalizationcited by 73
  15. 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
  16. 2021Exploring Memorization in Adversarial Trainingcited by 55
  17. 2020Part-dependent Label Noise: Towards Instance-dependent Label Noisecited by 40
  18. 2021Attacking Adversarial Attacks as A Defensecited by 31
  19. 2019Robust Local Features for Improving the Generalization of Adversarial Trainingcited by 30
  20. 2022Robust Weight Perturbation for Adversarial Trainingcited by 25
  21. 2021CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selectioncited by 21
  22. 2021Evaluating the Robustness of Geometry-Aware Instance-Reweighted Adversarial Trainingcited by 12
  23. 2021Guided Interpolation for Adversarial Trainingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 20; 2024: 11; 2025: 12; 2026: 4.

50 papers.

Sort:

By area:

  1. 2023Data Augmentation Alone Can Improve Adversarial Trainingcited by 43
  2. 2023Re-thinking Model Inversion Attacks Against Deep Neural Networkscited by 39
  3. 2024Few-Shot Adversarial Prompt Learning on Vision-Language Modelscited by 39
  4. 2023A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shotcited by 19
  5. 2023On the Over-Memorization During Natural, Robust and Catastrophic Overfittingcited by 19
  6. 2023Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspectivecited by 18
  7. 2023Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challengescited by 16
  8. 2023Adversarial Training Should Be Cast as a Non-Zero-Sum Gamecited by 13
  9. 2023Certified Robust Neural Networks: Generalization and Corruption Resistancecited by 11
  10. 2024Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularizationcited by 11
  11. 2023Robust Spatiotemporal Traffic Forecasting with Reinforced Dynamic Adversarial Trainingcited by 9
  12. 2023Initialization Matters for Adversarial Transfer Learningcited by 9
38 more
  1. 2024Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinementcited by 9
  2. 2024Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependencycited by 9
  3. 2024Shedding More Light on Robust Classifiers under the lens of Energy-based Modelscited by 9
  4. 2025AdLift: Lifting Adversarial Perturbations to Safeguard 3D Gaussian Splatting Assets Against Instruction-Driven Editingcited by 9
  5. 2023Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approachcited by 8
  6. 2023Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworkscited by 8
  7. 2023Data Optimization in Deep Learning: A Surveycited by 6
  8. 2024Understanding Model Ensemble in Transferable Adversarial Attackcited by 6
  9. 2025TAET: Two-Stage Adversarial Equalization Training on Long-Tailed Distributionscited by 6
  10. 2023On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamicscited by 5
  11. 2024Towards Adversarial Robustness via Debiased High-Confidence Logit Alignmentcited by 5
  12. 2025When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Needcited by 5
  13. 2022A3T: Accuracy Aware Adversarial Trainingcited by 4
  14. 2023Equalised Odds is not Equal Individual Odds: Post-processing for Group and Individual Fairnesscited by 4
  15. 2022Strength-Adaptive Adversarial Trainingcited by 3
  16. 2023Generating Less Certain Adversarial Examples Improves Robust Generalizationcited by 3
  17. 2024The Effectiveness of Random Forgetting for Robust Generalizationcited by 2
  18. 2025Rethinking Audio-Visual Adversarial Vulnerability from Temporal and Modality Perspectivescited by 2
  19. 2025Understanding Adversarial Training with Energy-based Modelscited by 2
  20. 2026LipNeXt: Scaling up Lipschitz-based Certified Robustness to Billion-parameter Modelscited by 2
  21. 2023Understanding Robust Overfitting from the Feature Generalization Perspectivecited by 1
  22. 2024Neural Networks with (Low-Precision) Polynomial Approximations: New Insights and Techniques for Accuracy Improvementcited by 1
  23. 2025Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environmentscited by 1
  24. 2025Parameter Interpolation Adversarial Training for Robust Image Classificationcited by 1
  25. 2026Does Order Matter : Connecting The Law of Robustness to Robust Generalizationcited by 1
  26. 2022Boundary Adversarial Examples Against Adversarial Overfittingcited by 0
  27. 2023PIAT: Parameter Interpolation based Adversarial Training for Image Classificationcited by 0
  28. 2023Adversarial Attacks Neutralization via Data Set Randomizationcited by 0
  29. 2023On Neural Network approximation of ideal adversarial attack and convergence of adversarial trainingcited by 0
  30. 2024Towards White Box Deep Learningcited by 0
  31. 2024Distributionally and Adversarially Robust Logistic Regression via Intersecting Wasserstein Ballscited by 0
  32. 2025Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimizationcited by 0
  33. 2025Feature Statistics with Uncertainty Help Adversarial Robustnesscited by 0
  34. 2025Identifying and Understanding Cross-Class Features in Adversarial Trainingcited by 0
  35. 2025ProARD: progressive adversarial robustness distillation: provide wide range of robust studentscited by 0
  36. 2025Stability and Generalization of Adversarial Diffusion Trainingcited by 0
  37. 2026Expanding the Role of Diffusion Models for Robust Classifier Trainingcited by 0
  38. 2026Robustness Emerges Early in Training Dynamics, but Is Not Preservedcited by 0