mlsanityData to 7 Oct 2026

Omnipotent Adversarial Training in the Wild

arXiv 2307.08596 ↗First posted 14 Jul 2023cs.LG cs.CR cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Adversarial training is an important topic in robust deep learning, but the community lacks attention to its practical usage. In this paper, we aim to resolve a real-world challenge, i.e., training a model on an imbalanced and noisy dataset to achieve high clean accuracy and adversarial robustness, with our proposed Omnipotent Adversarial Training (OAT) strategy. OAT consists of two innovative methodologies to address the imperfection in the training set. We first introduce an oracle into the adversarial training process to help the model learn a correct data-label conditional distribution. This carefully-designed oracle can provide correct label annotations for adversarial training. We further propose logits adjustment adversarial training to overcome the data imbalance issue, which can help the model learn a Bayes-optimal distribution. Our comprehensive evaluation results show that OAT outperforms other baselines by more than 20% clean accuracy improvement and 10% robust accuracy improvement under complex combinations of data imbalance and label noise scenarios. The code can be found in https://github.com/GuanlinLee/OAT.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

38 of 51 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  7. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  8. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  9. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  10. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  11. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  12. 2019When Does Label Smoothing Help?cited by 985
26 more
  1. 2017A Closer Look at Memorization in Deep Networkscited by 936
  2. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  3. 2019Adversarial Training for Free!cited by 638
  4. 2020Long-tail learning via logit adjustmentcited by 605
  5. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  6. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  7. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  8. 2020Overfitting in adversarially robust deep learningcited by 504
  9. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  10. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  11. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 397
  12. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  13. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  14. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  15. 2017Decoupling "when to update" from "how to update"cited by 230
  16. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  17. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  18. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  19. 2021Exploring Memorization in Adversarial Trainingcited by 55
  20. 2019Efficient Adversarial Training with Transferable Adversarial Examplescited by 54
  21. 2021Adversarial Robustness under Long-Tailed Distributioncited by 50
  22. 2014Training Convolutional Networks with Noisy Labelscited by 47
  23. 2019How does Early Stopping Help Generalization against Label Noise?cited by 41
  24. 2021Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfittingcited by 14
  25. 2021SSR: An Efficient and Robust Framework for Learning with Unknown Label Noisecited by 13
  26. 2020Towards Understanding the Regularization of Adversarial Robustness on Neural Networkscited by 12

Built on by

Not cited yet.