mlsanityData to 7 Oct 2026

Fast is better than free: Revisiting adversarial training

arXiv 2001.03994 ↗First posted 12 Jan 2020cs.LG stat.MLcited by 723

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with ε=8/255 in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at ε=2/255 in 12 hours, in comparison to past work based on "free" adversarial training which took 10 and 50 hours to reach the same respective thresholds. Finally, we identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at https://github.com/locuslab/fastadversarial.

Key facts. Cited by 723 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 161; 2022: 171; 2023: 159; 2024: 113; 2025: 79; 2026: 40.

723 papers.

Sort:

By area:

  1. 2022Diffusion Models for Adversarial Purificationcited by 469
  2. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  3. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  4. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  5. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  6. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  7. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  8. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  9. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  10. 2021Are Transformers More Robust Than CNNs?cited by 187
  11. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 164
  12. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
38 more
  1. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
  2. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  3. 2021On the Adversarial Robustness of Vision Transformerscited by 104
  4. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 104
  5. 2021Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attackscited by 101
  6. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 96
  7. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
  8. 2021Trustworthy AI: A Computational Perspectivecited by 93
  9. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  10. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  11. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 88
  12. 2023Robust Classification via a Single Diffusion Modelcited by 79
  13. 2024DiffuseMix: Label-Preserving Data Augmentation with Diffusion Modelscited by 78
  14. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  15. 2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
  16. 2022SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustnesscited by 74
  17. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  18. 2021On the effectiveness of adversarial training against common corruptionscited by 71
  19. 2021Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domaincited by 70
  20. 2021Skew Orthogonal Convolutionscited by 67
  21. 2021This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networkscited by 65
  22. 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
  23. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  24. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  25. 2023Structure Invariant Transformation for better Adversarial Transferabilitycited by 61
  26. 2021A Survey On Universal Adversarial Attackcited by 59
  27. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  28. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  29. 2021Exploring Memorization in Adversarial Trainingcited by 55
  30. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
  31. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 50
  32. 2021Adversarial Robustness under Long-Tailed Distributioncited by 50
  33. 2021Triangle Attack: A Query-efficient Decision-based Adversarial Attackcited by 48
  34. 2022AdvDO: Realistic Adversarial Attacks for Trajectory Predictioncited by 48
  35. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Modelscited by 48
  36. 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
  37. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  38. 2022Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?cited by 47

50 most cited of 723 All 723 (JSON)