mlsanityData to 14 Oct 2026

Stable and Efficient Adversarial Training through Local Linearization

arXiv 2210.05373 ↗First posted 11 Oct 2022cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

There has been a recent surge in single-step adversarial training as it shows robustness and efficiency. However, a phenomenon referred to as ``catastrophic overfitting" has been observed, which is prevalent in single-step defenses and may frustrate attempts to use FGSM adversarial training. To address this issue, we propose a novel method, Stable and Efficient Adversarial Training (SEAT), which mitigates catastrophic overfitting by harnessing on local properties that distinguish a robust model from that of a catastrophic overfitted model. The proposed SEAT has strong theoretical justifications, in that minimizing the SEAT loss can be shown to favour smooth empirical risk, thereby leading to robustness. Experimental results demonstrate that the proposed method successfully mitigates catastrophic overfitting, yielding superior performance amongst efficient defenses. Our single-step method can reach 51% robust accuracy for CIFAR-10 with l_∞ perturbations of radius 8/255 under a strong PGD-50 attack, matching the performance of a 10-step iterative adversarial training at merely 3% computational cost.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 46 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

37 of 46 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  9. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
25 more
  1. 2018Robustness May Be at Odds with Accuracycited by 880
  2. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  3. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  4. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  5. 2019Adversarial Training for Free!cited by 643
  6. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  7. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  8. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 414
  9. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  10. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 311
  11. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  12. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
  13. 2020Understanding and Improving Fast Adversarial Trainingcited by 214
  14. 2020Bag of Tricks for Adversarial Trainingcited by 200
  15. 2018Robustness via curvature regularization, and vice versacited by 175
  16. 2019Adversarial Robustness through Local Linearizationcited by 146
  17. 2021On the Convergence and Robustness of Adversarial Trainingcited by 130
  18. 2020Understanding Catastrophic Overfitting in Single-step Adversarial Trainingcited by 76
  19. 2020The Implicit and Explicit Regularization Effects of Dropoutcited by 71
  20. 2020Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defensescited by 62
  21. 2020Do We Need Zero Training Loss After Achieving Zero Training Error?cited by 61
  22. 2018Deep Defense: Training DNNs with Improved Adversarial Robustnesscited by 42
  23. 2020Single-step Adversarial training with Dropout Schedulingcited by 41
  24. 2021Subspace Adversarial Trainingcited by 41
  25. 2020Towards Understanding Fast Adversarial Trainingcited by 33

Built on by

Not cited yet.