# Towards Understanding Fast Adversarial Training

arXiv: https://arxiv.org/abs/2006.03089 · HTML: https://mlsanity.com/paper/2006.03089/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2020.

## Built on by (most cited first, 30 of 33)

- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Adversarial purification with Score-based generative models](https://mlsanity.com/paper/2106.06041/) (2021)
- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization](https://mlsanity.com/paper/2112.12376/) (2021)
- [Make Some Noise: Reliable and Efficient Single-Step Adversarial Training](https://mlsanity.com/paper/2202.01181/) (2022)
- [Reliably fast adversarial training via latent adversarial perturbation](https://mlsanity.com/paper/2104.01575/) (2021)
- [ZeroGrad : Mitigating and Explaining Catastrophic Overfitting in FGSM Adversarial Training](https://mlsanity.com/paper/2103.15476/) (2021)
- [Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart](https://mlsanity.com/paper/2105.14785/) (2021)
- [Why adversarial training can hurt robust accuracy](https://mlsanity.com/paper/2203.02006/) (2022)
- [Fast Adversarial Training with Smooth Convergence](https://mlsanity.com/paper/2308.12857/) (2023)
- [Understanding Catastrophic Overfitting in Adversarial Training](https://mlsanity.com/paper/2105.02942/) (2021)
- [Multi-objective Search of Robust Neural Architectures against Multiple Types of Adversarial Attacks](https://mlsanity.com/paper/2101.06507/) (2021)
- [Distributed Adversarial Training to Robustify Deep Neural Networks at Scale](https://mlsanity.com/paper/2206.06257/) (2022)
- [Efficient local linearity regularization to overcome catastrophic overfitting](https://mlsanity.com/paper/2401.11618/) (2024)
- [Style Curriculum Learning for Robust Medical Image Segmentation](https://mlsanity.com/paper/2108.00402/) (2021)
- [Robust Overfitting Does Matter: Test-Time Adversarial Purification With FGSM](https://mlsanity.com/paper/2403.11448/) (2024)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [Multi-stage Optimization based Adversarial Training](https://mlsanity.com/paper/2106.15357/) (2021)
- [Constrained Gradient Descent: A Powerful and Principled Evasion Attack Against Neural Networks](https://mlsanity.com/paper/2112.14232/) (2021)
- [Exploring Adversarial Robustness of Deep State Space Models](https://mlsanity.com/paper/2406.05532/) (2024)
- [Catastrophic overfitting can be induced with discriminative non-robust features](https://mlsanity.com/paper/2206.08242/) (2022)
- [Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectors](https://mlsanity.com/paper/2406.15104/) (2024)
- [Single-Step Adversarial Training for Semantic Segmentation](https://mlsanity.com/paper/2106.15998/) (2021)
- [Provable Robust Saliency-based Explanations](https://mlsanity.com/paper/2212.14106/) (2022)
- [Group-based Robustness: A General Framework for Customized Robustness in the Real World](https://mlsanity.com/paper/2306.16614/) (2023)
- [Robust Single-step Adversarial Training with Regularizer](https://mlsanity.com/paper/2102.03381/) (2021)
- [Lower Difficulty and Better Robustness: A Bregman Divergence Perspective for Adversarial Training](https://mlsanity.com/paper/2208.12511/) (2022)
- [Stable and Efficient Adversarial Training through Local Linearization](https://mlsanity.com/paper/2210.05373/) (2022)
- [Robust Ranking Explanations](https://mlsanity.com/paper/2307.04024/) (2023)

## Abstract (arXiv)

Current neural-network-based classifiers are susceptible to adversarial examples. The most empirically successful approach to defending against such adversarial examples is adversarial training, which incorporates a strong self-attack during training to enhance its robustness. This approach, however, is computationally expensive and hence is hard to scale up. A recent work, called fast adversarial training, has shown that it is possible to markedly reduce computation time without sacrificing significant performance. This approach incorporates simple self-attacks, yet it can only run for a limited number of training epochs, resulting in sub-optimal performance. In this paper, we conduct experiments to understand the behavior of fast adversarial training and show the key to its success is the ability to recover from overfitting to weak attacks. We then extend our findings to improve fast adversarial training, demonstrating superior robust accuracy to strong adversarial training, with much-reduced training time.