# On the effectiveness of adversarial training against common corruptions

arXiv: https://arxiv.org/abs/2103.02325 · HTML: https://mlsanity.com/paper/2103.02325/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 71 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 30 of 56)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Feature Denoising for Improving Adversarial Robustness](https://mlsanity.com/paper/1812.03411/) (2018)

## Built on by (most cited first, 30 of 71)

- [A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness](https://mlsanity.com/paper/2106.09129/) (2021)
- [PRIME: A few primitives can boost robustness to common corruptions](https://mlsanity.com/paper/2112.13547/) (2021)
- [Artificial Kuramoto Oscillatory Neurons](https://mlsanity.com/paper/2410.13821/) (2024)
- [Defending Against Image Corruptions Through Adversarial Augmentations](https://mlsanity.com/paper/2104.01086/) (2021)
- [Seasoning Model Soups for Robustness to Adversarial and Natural Distribution Shifts](https://mlsanity.com/paper/2302.10164/) (2023)
- [Improving Generalization of Adversarial Training via Robust Critical Fine-Tuning](https://mlsanity.com/paper/2308.02533/) (2023)
- [A Survey on the Robustness of Computer Vision Models against Common Corruptions](https://mlsanity.com/paper/2305.06024/) (2023)
- [CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters](https://mlsanity.com/paper/2203.15331/) (2022)
- [Revisiting adapters with adversarial training](https://mlsanity.com/paper/2210.04886/) (2022)
- [Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines](https://mlsanity.com/paper/2112.00659/) (2021)
- [Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors](https://mlsanity.com/paper/2210.06807/) (2022)
- [Adversarial Robustness for Tabular Data through Cost and Utility Awareness](https://mlsanity.com/paper/2208.13058/) (2022)
- [HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation](https://mlsanity.com/paper/2403.12033/) (2024)
- [Improving Robustness by Enhancing Weak Subnets](https://mlsanity.com/paper/2201.12765/) (2022)
- [Combining Diverse Feature Priors](https://mlsanity.com/paper/2110.08220/) (2021)
- [HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustness](https://mlsanity.com/paper/2307.11823/) (2023)
- [If a Human Can See It, So Should Your System: Reliability Requirements for Machine Vision Components](https://mlsanity.com/paper/2202.03930/) (2022)
- [Part-Based Models Improve Adversarial Robustness](https://mlsanity.com/paper/2209.09117/) (2022)
- [Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition](https://mlsanity.com/paper/2103.09947/) (2021)
- [Interpolated Joint Space Adversarial Training for Robust and Generalizable Defenses](https://mlsanity.com/paper/2112.06323/) (2021)
- [Benchmarking Robustness to Adversarial Image Obfuscations](https://mlsanity.com/paper/2301.12993/) (2023)
- [Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptation](https://mlsanity.com/paper/2404.10966/) (2024)
- [The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks](https://mlsanity.com/paper/2405.08886/) (2024)
- [Joint rotational invariance and adversarial training of a dual-stream Transformer yields state of the art Brain-Score for Area V4](https://mlsanity.com/paper/2203.06649/) (2022)
- [A Survey on AI Sustainability: Emerging Trends on Learning Algorithms and Research Challenges](https://mlsanity.com/paper/2205.03824/) (2022)
- [Benchmark Generation Framework with Customizable Distortions for Image Classifier Robustness](https://mlsanity.com/paper/2310.18626/) (2023)
- [Adversarially Robust Spiking Neural Networks Through Conversion](https://mlsanity.com/paper/2311.09266/) (2023)
- [Semantic Perturbations with Normalizing Flows for Improved Generalization](https://mlsanity.com/paper/2108.07958/) (2021)
- [How Robust are Discriminatively Trained Zero-Shot Learning Models?](https://mlsanity.com/paper/2201.10972/) (2022)
- [AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentation](https://mlsanity.com/paper/2306.07197/) (2023)

## Abstract (arXiv)

The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when used with an appropriately selected perturbation radius, ℓp adversarial training can serve as a strong baseline against common corruptions improving both accuracy and calibration. Then we explain why adversarial training performs better than data augmentation with simple Gaussian noise which has been observed to be a meaningful baseline on common corruptions. Related to this, we identify the σ-overfitting phenomenon when Gaussian augmentation overfits to a particular standard deviation used for training which has a significant detrimental effect on common corruption accuracy. We discuss how to alleviate this problem and then how to further enhance ℓp adversarial training by introducing an efficient relaxation of adversarial training with learned perceptual image patch similarity as the distance metric. Through experiments on CIFAR-10 and ImageNet-100, we show that our approach does not only improve the ℓp adversarial training baseline but also has cumulative gains with data augmentation methods such as AugMix, DeepAugment, ANT, and SIN, leading to state-of-the-art performance on common corruptions. The code of our experiments is publicly available at https://github.com/tml-epfl/adv-training-corruptions.