mlsanityData to 7 Oct 2026

Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement

arXiv 2403.09101 ↗First posted 14 Mar 2024cs.LG cs.CR cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Adversarial training (AT) is currently one of the most effective ways to obtain the robustness of deep neural networks against adversarial attacks. However, most AT methods suffer from robust overfitting, i.e., a significant generalization gap in adversarial robustness between the training and testing curves. In this paper, we first identify a connection between robust overfitting and the excessive memorization of noisy labels in AT from a view of gradient norm. As such label noise is mainly caused by a distribution mismatch and improper label assignments, we are motivated to propose a label refinement approach for AT. Specifically, our Self-Guided Label Refinement first self-refines a more accurate and informative label distribution from over-confident hard labels, and then it calibrates the training by dynamically incorporating knowledge from self-distilled models into the current model and thus requiring no external teachers. Empirical results demonstrate that our method can simultaneously boost the standard accuracy and robust performance across multiple benchmark datasets, attack types, and architectures. In addition, we also provide a set of analyses from the perspectives of information theory to dive into our method and suggest the importance of soft labels for robust generalization.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 33 of its 41 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

33 of 41 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  12. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
21 more
  1. 2020Overfitting in adversarially robust deep learningcited by 504
  2. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  3. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  4. 2019On Evaluating Adversarial Robustnesscited by 442
  5. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  6. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  7. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 213
  8. 2020Does label smoothing mitigate label noise?cited by 204
  9. 2020Bag of Tricks for Adversarial Trainingcited by 199
  10. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  11. 2019Adversarially Robust Distillationcited by 124
  12. 2020Rethinking Bias-Variance Trade-off for Generalization of Neural Networkscited by 107
  13. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 77
  14. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  15. 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 50
  16. 2021On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implicationscited by 31
  17. 2021PAC-Bayes Information Bottleneckcited by 28
  18. 2021Mitigating Memorization of Noisy Labels via Regularization between Representationscited by 26
  19. 2022Robust Weight Perturbation for Adversarial Trainingcited by 25
  20. 2023Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspectivecited by 18
  21. 2021Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfittingcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 2.

9 papers.

  1. 2024Self-Improving Robust Preference Optimizationcited by 19
  2. 2024Adversarial Training: A Surveycited by 6
  3. 2024Towards Adversarial Robustness via Debiased High-Confidence Logit Alignmentcited by 5
  4. 2024Mitigating Low-Frequency Bias: Feature Recalibration and Frequency Attention Regularization for Adversarial Robustnesscited by 3
  5. 2025Unsupervised Robust Domain Adaptation: Paradigm, Theory and Algorithmcited by 3
  6. 2025Your Classifier Can Do More: Towards Balancing the Gaps in Classification, Robustness, and Generationcited by 1
  7. 2025The Power of Many: Synergistic Unification of Diverse Augmentations for Efficient Adversarial Robustnesscited by 0
  8. 2026Adaptive Causal Alignment for High-Confidence Adversarial Trainingcited by 0
  9. 2026Improving Adversarial Robustness via Activation Amplification and Attenuationcited by 0