mlsanityData to 7 Oct 2026

Understanding the Interaction of Adversarial Training with Noisy Labels

arXiv 2102.03482 ↗First posted 6 Feb 2021cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Noisy labels (NL) and adversarial examples both undermine trained models, but interestingly they have hitherto been studied independently. A recent adversarial training (AT) study showed that the number of projected gradient descent (PGD) steps to successfully attack a point (i.e., find an adversarial example in its proximity) is an effective measure of the robustness of this point. Given that natural data are clean, this measure reveals an intrinsic geometric property -- how far a point is from its class boundary. Based on this breakthrough, in this paper, we figure out how AT would interact with NL. Firstly, we find if a point is too close to its noisy-class boundary (e.g., one step is enough to attack it), this point is likely to be mislabeled, which suggests to adopt the number of PGD steps as a new criterion for sample selection for correcting NL. Secondly, we confirm AT with strong smoothing effects suffers less from NL (without NL corrections) than standard training (ST), which suggests AT itself is an NL correction. Hence, AT with NL is helpful for improving even the natural accuracy, which again illustrates the superiority of AT as a general-purpose robust learning criterion.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 6 Feb 2021.

Builds on

35 of 56 references.

References · 35

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2016Wide Residual Networkscited by 3,057
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  10. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  11. 2017A Closer Look at Memorization in Deep Networkscited by 936
  12. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
23 more
  1. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  2. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  3. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  4. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  5. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  6. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  7. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  8. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  9. 2017Decoupling "when to update" from "how to update"cited by 230
  10. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  11. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  12. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  13. 2019Are Labels Required for Improving Adversarial Robustness?cited by 162
  14. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  15. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  16. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  17. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  18. 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
  19. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  20. 2016Towards the Science of Security and Privacy in Machine Learningcited by 89
  21. 2018Curriculum Adversarial Trainingcited by 89
  22. 2020Training Binary Neural Networks through Learning with Noisy Supervisioncited by 39
  23. 2019Where is the Bottleneck of Adversarial Learning with Unlabeled Data?cited by 3

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 3; 2023: 7; 2024: 3; 2025: 2.

19 papers.

Sort:

  1. 2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 58
  2. 2022Can Adversarial Training Be Manipulated By Non-Robust Features?cited by 12
  3. 2025Accurate Forgetting for Heterogeneous Federated Continual Learningcited by 11
  4. 2023Outlier Robust Adversarial Trainingcited by 10
  5. 2023It Is All About Data: A Survey on the Effects of Data on Adversarial Robustnesscited by 9
  6. 2023Combating Bilateral Edge Noise for Robust Link Predictioncited by 9
  7. 2021NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labelscited by 8
  8. 2022A law of adversarial risk, interpolation, and label noisecited by 8
  9. 2024Boosting Model Resilience via Implicit Adversarial Data Augmentationcited by 7
  10. 2023Combining Adversaries with Anti-adversaries in Trainingcited by 6
  11. 2025When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Needcited by 5
  12. 2021Multiplicative Reweighting for Robust Neural Network Optimizationcited by 4
7 more
  1. 2022Adversarial Training with Complementary Labels: On the Benefit of Gradually Informative Attackscited by 4
  2. 2023MILD: Modeling the Instance Learning Dynamics for Learning with Noisy Labelscited by 3
  3. 2023Collapse-Aware Triplet Decoupling for Adversarially Robust Image Retrievalcited by 2
  4. 2024Informed Deep Abstaining Classifier: Investigating noise-robust training for diagnostic decision support systemscited by 2
  5. 2021Local Reweighting for Adversarial Trainingcited by 1
  6. 2024Revisiting Semi-supervised Adversarial Robustness via Noise-aware Online Robust Distillationcited by 1
  7. 2023A Data-Centric Approach for Improving Adversarial Training Through the Lens of Out-of-Distribution Detectioncited by 0