# Efficient Adversarial Training with Transferable Adversarial Examples

arXiv: https://arxiv.org/abs/1912.11969 · HTML: https://mlsanity.com/paper/1912.11969/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2019.

## Built on by (most cited first, 30 of 56)

- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization](https://mlsanity.com/paper/2112.12376/) (2021)
- [SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness](https://mlsanity.com/paper/2207.12391/) (2022)
- [Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples](https://mlsanity.com/paper/2307.10562/) (2023)
- [Fast Adversarial Training with Adaptive Step Size](https://mlsanity.com/paper/2206.02417/) (2022)
- [Meta Adversarial Training against Universal Patches](https://mlsanity.com/paper/2101.11453/) (2021)
- [REAP: A Large-Scale Realistic Adversarial Patch Benchmark](https://mlsanity.com/paper/2212.05680/) (2022)
- [Improving the Accuracy-Robustness Trade-Off of Classifiers via Adaptive Smoothing](https://mlsanity.com/paper/2301.12554/) (2023)
- [Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging](https://mlsanity.com/paper/2308.11443/) (2023)
- [Group-wise Inhibition based Feature Regularization for Robust Classification](https://mlsanity.com/paper/2103.02152/) (2021)
- [On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training](https://mlsanity.com/paper/2112.07324/) (2021)
- [Robust Real-World Image Super-Resolution against Adversarial Attacks](https://mlsanity.com/paper/2208.00428/) (2022)
- [Robustness through Data Augmentation Loss Consistency](https://mlsanity.com/paper/2110.11205/) (2021)
- [FedRight: An Effective Model Copyright Protection for Federated Learning](https://mlsanity.com/paper/2303.10399/) (2023)
- [Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness](https://mlsanity.com/paper/2112.00323/) (2021)
- [SRoUDA: Meta Self-training for Robust Unsupervised Domain Adaptation](https://mlsanity.com/paper/2212.05917/) (2022)
- [Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensation](https://mlsanity.com/paper/2310.07506/) (2023)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines](https://mlsanity.com/paper/2112.09219/) (2021)
- [Robust Binary Models by Pruning Randomly-initialized Networks](https://mlsanity.com/paper/2202.01341/) (2022)
- [Revisiting Outer Optimization in Adversarial Training](https://mlsanity.com/paper/2209.01199/) (2022)
- [Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective](https://mlsanity.com/paper/2302.11963/) (2023)
- [Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey](https://mlsanity.com/paper/2303.06302/) (2023)
- [Universal Adversarial Defense in Remote Sensing Based on Pre-trained Denoising Diffusion Models](https://mlsanity.com/paper/2307.16865/) (2023)
- [Random Walks for Adversarial Meshes](https://mlsanity.com/paper/2202.07453/) (2022)
- [Adversarial Attacks on Speech Recognition Systems for Mission-Critical Applications: A Survey](https://mlsanity.com/paper/2202.10594/) (2022)
- [A Girl Has A Name, And It's ... Adversarial Authorship Attribution for Deobfuscation](https://mlsanity.com/paper/2203.11849/) (2022)
- [Batch Normalization Is Blind to the First and Second Derivatives of the Loss](https://mlsanity.com/paper/2205.15146/) (2022)
- [Hard Adversarial Example Mining for Improving Robust Fairness](https://mlsanity.com/paper/2308.01823/) (2023)
- [Raze to the Ground: Query-Efficient Adversarial HTML Attacks on Machine-Learning Phishing Webpage Detectors](https://mlsanity.com/paper/2310.03166/) (2023)

## Abstract (arXiv)

Adversarial training is an effective defense method to protect classification models against adversarial attacks. However, one limitation of this approach is that it can require orders of magnitude additional training time due to high cost of generating strong adversarial examples during training. In this paper, we first show that there is high transferability between models from neighboring epochs in the same training process, i.e., adversarial examples from one epoch continue to be adversarial in subsequent epochs. Leveraging this property, we propose a novel method, Adversarial Training with Transferable Adversarial Examples (ATTA), that can enhance the robustness of trained models and greatly improve the training efficiency by accumulating adversarial perturbations through epochs. Compared to state-of-the-art adversarial training methods, ATTA enhances adversarial accuracy by up to 7.2% on CIFAR10 and requires 12~14x less training time on MNIST and CIFAR10 datasets with comparable model robustness.