# Improving the affordability of robustness training for DNNs

arXiv: https://arxiv.org/abs/2002.04237 · HTML: https://mlsanity.com/paper/2002.04237/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 8 of 8)

- [Adversarial training in communication constrained federated learning](https://mlsanity.com/paper/2103.01319/) (2021)
- [Towards Robust General Medical Image Segmentation](https://mlsanity.com/paper/2107.04263/) (2021)
- [Demystifying the Adversarial Robustness of Random Transformation Defenses](https://mlsanity.com/paper/2207.03574/) (2022)
- [Part-Based Models Improve Adversarial Robustness](https://mlsanity.com/paper/2209.09117/) (2022)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Hyper-parameter Tuning for Adversarially Robust Models](https://mlsanity.com/paper/2304.02497/) (2023)
- [ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise](https://mlsanity.com/paper/2201.12469/) (2022)
- [Reducing Adversarial Training Cost with Gradient Approximation](https://mlsanity.com/paper/2309.09464/) (2023)

## Abstract (arXiv)

Projected Gradient Descent (PGD) based adversarial training has become one of the most prominent methods for building robust deep neural network models. However, the computational complexity associated with this approach, due to the maximization of the loss function when finding adversaries, is a longstanding problem and may be prohibitive when using larger and more complex models. In this paper we show that the initial phase of adversarial training is redundant and can be replaced with natural training which significantly improves the computational efficiency. We demonstrate that this efficiency gain can be achieved without any loss in accuracy on natural and adversarial test samples. We support our argument with insights on the nature of the adversaries and their relative strength during the training process. We show that our proposed method can reduce the training time by a factor of up to 2.5 with comparable or better model test accuracy and generalization on various strengths of adversarial attacks.