# Prior-Guided Adversarial Initialization for Fast Adversarial Training

arXiv: https://arxiv.org/abs/2207.08859 · HTML: https://mlsanity.com/paper/2207.08859/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 54 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack](https://mlsanity.com/paper/1907.02044/) (2019)
- [Enhancing the Transferability of Adversarial Attacks through Variance Tuning](https://mlsanity.com/paper/2103.15571/) (2021)
- [Adversarial Logit Pairing](https://mlsanity.com/paper/1803.06373/) (2018)
- [Understanding and Improving Fast Adversarial Training](https://mlsanity.com/paper/2007.02617/) (2020)
- [Bag of Tricks for Adversarial Training](https://mlsanity.com/paper/2010.00467/) (2020)
- [You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle](https://mlsanity.com/paper/1905.00877/) (2019)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [On the Convergence and Robustness of Adversarial Training](https://mlsanity.com/paper/2112.08304/) (2021)
- [Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles](https://mlsanity.com/paper/2003.08757/) (2020)
- [Transferable Adversarial Attacks for Image and Video Object Detection](https://mlsanity.com/paper/1811.12641/) (2018)
- [ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples](https://mlsanity.com/paper/1811.12673/) (2018)
- [Boosting Adversarial Training with Hypersphere Embedding](https://mlsanity.com/paper/2002.08619/) (2020)
- [LAS-AT: Adversarial Training with Learnable Attack Strategy](https://mlsanity.com/paper/2203.06616/) (2022)
- [Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object Detection](https://mlsanity.com/paper/2201.08970/) (2022)

## Built on by (most cited first, 30 of 35)

- [Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples](https://mlsanity.com/paper/2307.10562/) (2023)
- [OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization](https://mlsanity.com/paper/2312.04403/) (2023)
- [Improving Fast Adversarial Training with Prior-Guided Knowledge](https://mlsanity.com/paper/2304.00202/) (2023)
- [Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentation](https://mlsanity.com/paper/2209.09841/) (2022)
- [Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Network](https://mlsanity.com/paper/2412.15678/) (2024)
- [Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging](https://mlsanity.com/paper/2308.11443/) (2023)
- [Fast Adversarial Training with Smooth Convergence](https://mlsanity.com/paper/2308.12857/) (2023)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective](https://mlsanity.com/paper/2407.12443/) (2024)
- [Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularization](https://mlsanity.com/paper/2404.08154/) (2024)
- [Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks](https://mlsanity.com/paper/2310.15444/) (2023)
- [Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation](https://mlsanity.com/paper/2312.05508/) (2023)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective](https://mlsanity.com/paper/2302.11963/) (2023)
- [AEGIS: Adversarial Target-Guided Retention-Data-Free Robust Concept Erasure from Diffusion Models](https://mlsanity.com/paper/2602.06771/) (2026)
- [Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filters](https://mlsanity.com/paper/2306.03430/) (2023)
- [Catastrophic Overfitting: A Potential Blessing in Disguise](https://mlsanity.com/paper/2402.18211/) (2024)
- [Improving Fast Adversarial Training via Self-Knowledge Guidance](https://mlsanity.com/paper/2409.17589/) (2024)
- [Your Classifier Can Do More: Towards Balancing the Gaps in Classification, Robustness, and Generation](https://mlsanity.com/paper/2505.19459/) (2025)
- [FastAT Benchmark: A Comprehensive Framework for Fair Evaluation of Fast Adversarial Training Methods](https://mlsanity.com/paper/2604.22853/) (2026)
- [Mitigating Error Amplification in Fast Adversarial Training](https://mlsanity.com/paper/2604.24332/) (2026)
- [A Spectral Perspective towards Understanding and Improving Adversarial Robustness](https://mlsanity.com/paper/2306.14262/) (2023)
- [General Lipschitz: Certified Robustness Against Resolvable Semantic Transformations via Transformation-Dependent Randomized Smoothing](https://mlsanity.com/paper/2309.16710/) (2023)
- [Towards Understanding Dual BN In Hybrid Adversarial Training](https://mlsanity.com/paper/2403.19150/) (2024)
- [Improving Fast Adversarial Training Paradigm: An Example Taxonomy Perspective](https://mlsanity.com/paper/2408.03944/) (2024)
- [Towards Class-wise Fair Adversarial Training via Anti-Bias Soft Label Distillation](https://mlsanity.com/paper/2506.08611/) (2025)
- [Are Fast Methods Stable in Adversarially Robust Transfer Learning?](https://mlsanity.com/paper/2506.22602/) (2025)
- [TRIX- Trading Adversarial Fairness via Mixed Adversarial Training](https://mlsanity.com/paper/2507.07768/) (2025)
- [RegMix: Adversarial Mutual and Generalization Regularization for Enhancing DNN Robustness](https://mlsanity.com/paper/2510.05317/) (2025)
- [Lipschitz-aware Linearity Grafting for Certified Robustness](https://mlsanity.com/paper/2510.25130/) (2025)

## Abstract (arXiv)

Fast adversarial training (FAT) effectively improves the efficiency of standard adversarial training (SAT). However, initial FAT encounters catastrophic overfitting, i.e.,the robust accuracy against adversarial attacks suddenly and dramatically decreases. Though several FAT variants spare no effort to prevent overfitting, they sacrifice much calculation cost. In this paper, we explore the difference between the training processes of SAT and FAT and observe that the attack success rate of adversarial examples (AEs) of FAT gets worse gradually in the late training stage, resulting in overfitting. The AEs are generated by the fast gradient sign method (FGSM) with a zero or random initialization. Based on the observation, we propose a prior-guided FGSM initialization method to avoid overfitting after investigating several initialization strategies, improving the quality of the AEs during the whole training process. The initialization is formed by leveraging historically generated AEs without additional calculation cost. We further provide a theoretical analysis for the proposed initialization method. We also propose a simple yet effective regularizer based on the prior-guided initialization,i.e., the currently generated perturbation should not deviate too much from the prior-guided initialization. The regularizer adopts both historical and current adversarial perturbations to guide the model learning. Evaluations on four datasets demonstrate that the proposed method can prevent catastrophic overfitting and outperform state-of-the-art FAT methods. The code is released at https://github.com/jiaxiaojunQAQ/FGSM-PGI.