# ℓ_∞-Robustness and Beyond: Unleashing Efficient Adversarial Training

arXiv: https://arxiv.org/abs/2112.00378 · HTML: https://mlsanity.com/paper/2112.00378/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training](https://mlsanity.com/paper/2103.00123/) (2021)
- [GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning](https://mlsanity.com/paper/2012.10630/) (2020)
- [Understanding and Improving Fast Adversarial Training](https://mlsanity.com/paper/2007.02617/) (2020)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [Perceptual Adversarial Robustness: Defense Against Unseen Threat Models](https://mlsanity.com/paper/2006.12655/) (2020)
- [Functional Adversarial Attacks](https://mlsanity.com/paper/1906.00001/) (2019)
- [Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent](https://mlsanity.com/paper/1802.01737/) (2018)
- [Restricted Strong Convexity Implies Weak Submodularity](https://mlsanity.com/paper/1612.00804/) (2016)
- [Introduction to Core-sets: an Updated Survey](https://mlsanity.com/paper/2011.09384/) (2020)

## Built on by (most cited first, 11 of 11)

- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning](https://mlsanity.com/paper/2311.13613/) (2023)
- [MimicDiffusion: Purifying Adversarial Perturbation via Mimicking Clean Diffusion Model](https://mlsanity.com/paper/2312.04802/) (2023)
- [Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization](https://mlsanity.com/paper/2401.16352/) (2024)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Adversarial Guided Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2403.16067/) (2024)
- [Training-Free Dataset Pruning for Instance Segmentation](https://mlsanity.com/paper/2503.00828/) (2025)
- [Diffusion-based Cumulative Adversarial Purification for Vision Language Models](https://mlsanity.com/paper/2506.03933/) (2025)
- [Do we need entire training data for adversarial training?](https://mlsanity.com/paper/2303.06241/) (2023)
- [LightPure: Realtime Adversarial Image Purification for Mobile Devices Using Diffusion Models](https://mlsanity.com/paper/2409.00340/) (2024)
- [Model-Free Adversarial Purification via Coarse-To-Fine Tensor Network Representation](https://mlsanity.com/paper/2502.17972/) (2025)

## Abstract (arXiv)

Neural networks are vulnerable to adversarial attacks: adding well-crafted, imperceptible perturbations to their input can modify their output. Adversarial training is one of the most effective approaches in training robust models against such attacks. However, it is much slower than vanilla training of neural networks since it needs to construct adversarial examples for the entire training data at every iteration, hampering its effectiveness. Recently, Fast Adversarial Training (FAT) was proposed that can obtain robust models efficiently. However, the reasons behind its success are not fully understood, and more importantly, it can only train robust models for ℓ_∞-bounded attacks as it uses FGSM during training. In this paper, by leveraging the theory of coreset selection, we show how selecting a small subset of training data provides a general, more principled approach toward reducing the time complexity of robust training. Unlike existing methods, our approach can be adapted to a wide variety of training objectives, including TRADES, ℓp-PGD, and Perceptual Adversarial Training (PAT). Our experimental results indicate that our approach speeds up adversarial training by 2-3 times while experiencing a slight reduction in the clean and robust accuracy.