mlsanityData to 14 Oct 2026

Multi-stage Optimization based Adversarial Training

arXiv 2106.15357 ↗First posted 26 Jun 2021cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In the field of adversarial robustness, there is a common practice that adopts the single-step adversarial training for quickly developing adversarially robust models. However, the single-step adversarial training is most likely to cause catastrophic overfitting, as after a few training epochs it will be hard to generate strong adversarial examples to continuously boost the adversarial robustness. In this work, we aim to avoid the catastrophic overfitting by introducing multi-step adversarial examples during the single-step adversarial training. Then, to balance the large training overhead of generating multi-step adversarial examples, we propose a Multi-stage Optimization based Adversarial Training (MOAT) method that periodically trains the model on mixed benign examples, single-step adversarial examples, and multi-step adversarial examples stage by stage. In this way, the overall training overhead is reduced significantly, meanwhile, the model could avoid catastrophic overfitting. Extensive experiments on CIFAR-10 and CIFAR-100 datasets demonstrate that under similar amount of training overhead, the proposed MOAT exhibits better robustness than either single-step or multi-step adversarial training methods.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 44 of its 54 references that are in the map. First posted on arXiv 26 Jun 2021.

Builds on

44 of 54 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2013Intriguing properties of neural networkscited by 5,155
  7. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  8. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  9. 2015Fast R-CNNcited by 2,933
  10. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  11. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  12. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
32 more
  1. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  3. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  4. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  5. 2019Adversarial Training for Free!cited by 643
  6. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  7. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 599
  8. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
  9. 2017Countering Adversarial Images using Input Transformationscited by 520
  10. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 501
  11. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  12. 2017Mitigating Adversarial Effects Through Randomizationcited by 383
  13. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  14. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  15. 2018Certified Defenses against Adversarial Examplescited by 306
  16. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 244
  17. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 244
  18. 2020Understanding and Improving Fast Adversarial Trainingcited by 214
  19. 2017Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Ratescited by 199
  20. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 183
  21. 2018Semidefinite relaxations for certifying robustness to adversarial examplescited by 179
  22. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 172
  23. 2018Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priorscited by 146
  24. 2018Query-Efficient Hard-label Black-box Attack:An Optimization-based Approachcited by 130
  25. 2019Improving Black-box Adversarial Attacks with a Transfer-based Priorcited by 121
  26. 2020QEBA: Query-Efficient Boundary-Based Blackbox Attackcited by 104
  27. 2020Understanding Catastrophic Overfitting in Single-step Adversarial Trainingcited by 76
  28. 2018Adversarial Attacks Against Medical Deep Learning Systemscited by 62
  29. 2018Improving the Generalization of Adversarial Training with Domain Adaptationcited by 48
  30. 2020Single-step Adversarial training with Dropout Schedulingcited by 41
  31. 2019Subspace Attack: Exploiting Promising Subspaces for Query-Efficient Black-box Attackscited by 35
  32. 2020Towards Understanding Fast Adversarial Trainingcited by 33

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2023Diversifying the High-level Features for better Adversarial Transferabilitycited by 34
  2. 2024Learning to Transform Dynamically for Better Adversarial Transferabilitycited by 23
  3. 2024Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularizationcited by 12
  4. 2025Enhancing Adversarial Transferability via Component-Wise Transformationcited by 1