# Subspace Adversarial Training

arXiv: https://arxiv.org/abs/2111.12229 · HTML: https://mlsanity.com/paper/2111.12229/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Provable defenses against adversarial examples via the convex outer adversarial polytope](https://mlsanity.com/paper/1711.00851/) (2017)
- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [Fantastic Generalization Measures and Where to Find Them](https://mlsanity.com/paper/1912.02178/) (2019)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples](https://mlsanity.com/paper/1710.10766/) (2017)
- [Attacks Which Do Not Kill Training Make Adversarial Learning Stronger](https://mlsanity.com/paper/2002.11242/) (2020)
- [Gradient Descent Happens in a Tiny Subspace](https://mlsanity.com/paper/1812.04754/) (2018)
- [Understanding and Improving Fast Adversarial Training](https://mlsanity.com/paper/2007.02617/) (2020)
- [Robustness via curvature regularization, and vice versa](https://mlsanity.com/paper/1811.09716/) (2018)
- [Implicit Gradient Regularization](https://mlsanity.com/paper/2009.11162/) (2020)
- [Understanding Catastrophic Overfitting in Single-step Adversarial Training](https://mlsanity.com/paper/2010.01799/) (2020)

## Built on by (most cited first, 30 of 41)

- [Efficient and Effective Augmentation Strategy for Adversarial Training](https://mlsanity.com/paper/2210.15318/) (2022)
- [Improving Fast Adversarial Training with Prior-Guided Knowledge](https://mlsanity.com/paper/2304.00202/) (2023)
- [One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks](https://mlsanity.com/paper/2205.12141/) (2022)
- [Fast Adversarial Training with Adaptive Step Size](https://mlsanity.com/paper/2206.02417/) (2022)
- [Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape](https://mlsanity.com/paper/2409.14396/) (2024)
- [Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attacks](https://mlsanity.com/paper/2205.12134/) (2022)
- [Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models](https://mlsanity.com/paper/2305.17446/) (2023)
- [Subspace based Federated Unlearning](https://mlsanity.com/paper/2302.12448/) (2023)
- [Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging](https://mlsanity.com/paper/2308.11443/) (2023)
- [Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularization](https://mlsanity.com/paper/2404.08154/) (2024)
- [Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency](https://mlsanity.com/paper/2405.16262/) (2024)
- [Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks](https://mlsanity.com/paper/2310.15444/) (2023)
- [Efficient local linearity regularization to overcome catastrophic overfitting](https://mlsanity.com/paper/2401.11618/) (2024)
- [RobArch: Designing Robust Architectures against Adversarial Attacks](https://mlsanity.com/paper/2301.03110/) (2023)
- [Robust Overfitting Does Matter: Test-Time Adversarial Purification With FGSM](https://mlsanity.com/paper/2403.11448/) (2024)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective](https://mlsanity.com/paper/2302.11963/) (2023)
- [Identifying Policy Gradient Subspaces](https://mlsanity.com/paper/2401.06604/) (2024)
- [Catastrophic Overfitting: A Potential Blessing in Disguise](https://mlsanity.com/paper/2402.18211/) (2024)
- [Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization](https://mlsanity.com/paper/2211.05554/) (2022)
- [Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy](https://mlsanity.com/paper/2407.03641/) (2024)
- [On Using Certified Training towards Empirical Robustness](https://mlsanity.com/paper/2410.01617/) (2024)
- [The Effectiveness of Random Forgetting for Robust Generalization](https://mlsanity.com/paper/2402.11733/) (2024)
- [Edge Detectors Can Make Deep Convolutional Neural Networks More Robust](https://mlsanity.com/paper/2402.16479/) (2024)
- [Improving Fast Adversarial Training via Self-Knowledge Guidance](https://mlsanity.com/paper/2409.17589/) (2024)
- [Understanding Adversarial Training with Energy-based Models](https://mlsanity.com/paper/2505.22486/) (2025)
- [Unifying Gradients to Improve Real-world Robustness for Deep Networks](https://mlsanity.com/paper/2208.06228/) (2022)
- [Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation](https://mlsanity.com/paper/2311.17607/) (2023)
- [ZeroPur: Succinct Training-Free Adversarial Purification](https://mlsanity.com/paper/2406.03143/) (2024)
- [Label Augmentation for Neural Networks Robustness](https://mlsanity.com/paper/2408.01977/) (2024)

## Abstract (arXiv)

Single-step adversarial training (AT) has received wide attention as it proved to be both efficient and robust. However, a serious problem of catastrophic overfitting exists, i.e., the robust accuracy against projected gradient descent (PGD) attack suddenly drops to 0% during the training. In this paper, we approach this problem from a novel perspective of optimization and firstly reveal the close link between the fast-growing gradient of each sample and overfitting, which can also be applied to understand robust overfitting in multi-step AT. To control the growth of the gradient, we propose a new AT method, Subspace Adversarial Training (Sub-AT), which constrains AT in a carefully extracted subspace. It successfully resolves both kinds of overfitting and significantly boosts the robustness. In subspace, we also allow single-step AT with larger steps and larger radius, further improving the robustness performance. As a result, we achieve state-of-the-art single-step AT performance. Without any regularization term, our single-step AT can reach over 51% robust accuracy against strong PGD-50 attack of radius 8/255 on CIFAR-10, reaching a competitive performance against standard multi-step PGD-10 AT with huge computational advantages. The code is released at https://github.com/nblt/Sub-AT.