mlsanityData to 7 Oct 2026

On the effectiveness of adversarial training against common corruptions

arXiv 2103.02325 ↗First posted 3 Mar 2021cs.LG cs.AI cs.CVcited by 71

Not on the map. Why?

Abstract (arXiv)

The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when used with an appropriately selected perturbation radius, ℓp adversarial training can serve as a strong baseline against common corruptions improving both accuracy and calibration. Then we explain why adversarial training performs better than data augmentation with simple Gaussian noise which has been observed to be a meaningful baseline on common corruptions. Related to this, we identify the σ-overfitting phenomenon when Gaussian augmentation overfits to a particular standard deviation used for training which has a significant detrimental effect on common corruption accuracy. We discuss how to alleviate this problem and then how to further enhance ℓp adversarial training by introducing an efficient relaxation of adversarial training with learned perceptual image patch similarity as the distance metric. Through experiments on CIFAR-10 and ImageNet-100, we show that our approach does not only improve the ℓp adversarial training baseline but also has cumulative gains with data augmentation methods such as AugMix, DeepAugment, ANT, and SIN, leading to state-of-the-art performance on common corruptions. The code of our experiments is publicly available at https://github.com/tml-epfl/adv-training-corruptions.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 56 of its 71 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

56 of 71 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2017On Calibration of Modern Neural Networkscited by 4,860
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
38 more
  1. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  2. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  3. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  4. 2019Natural Adversarial Examplescited by 1,401
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  6. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  7. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  8. 2018Robustness May Be at Odds with Accuracycited by 873
  9. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  10. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  11. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  12. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  13. 2019Adversarial Training for Free!cited by 638
  14. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  15. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  16. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  17. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  18. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  19. 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
  20. 2019Adversarial Examples Improve Image Recognitioncited by 283
  21. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  22. 2018Generalisation in humans and deep neural networkscited by 237
  23. 2018Adversarial Logit Pairingcited by 224
  24. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  25. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  26. 2018Robustness via curvature regularization, and vice versacited by 174
  27. 2018Disentangling Adversarial Robustness and Generalizationcited by 171
  28. 2019Adversarial Examples Are a Natural Consequence of Test Error in Noisecited by 114
  29. 2017A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortionscited by 107
  30. 2021On Interaction Between Augmentations and Corruptions in Natural Corruption Robustnesscited by 98
  31. 2018Universal Adversarial Trainingcited by 85
  32. 2018On the Suitability of Lp-norms for Creating and Preventing Adversarial Examplescited by 57
  33. 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
  34. 2020Learning perturbation sets for robust machine learningcited by 48
  35. 2018Evaluating and Understanding the Robustness of Adversarial Logit Pairingcited by 34
  36. 2015Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimizationcited by 32
  37. 2020Hold me tight! Influence of discriminative features on deep network boundariescited by 31
  38. 2018Logit Pairing Methods Can Fool Gradient-Based Attackscited by 27

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 20; 2023: 24; 2024: 10; 2025: 6; 2026: 2.

71 papers.

Sort:

By area:

  1. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  2. 2021PRIME: A few primitives can boost robustness to common corruptionscited by 40
  3. 2024Artificial Kuramoto Oscillatory Neuronscited by 37
  4. 2021Defending Against Image Corruptions Through Adversarial Augmentationscited by 28
  5. 2023Seasoning Model Soups for Robustness to Adversarial and Natural Distribution Shiftscited by 23
  6. 2023Improving Generalization of Adversarial Training via Robust Critical Fine-Tuningcited by 23
  7. 2022CNN Filter DB: An Empirical Investigation of Trained Convolutional Filterscited by 20
  8. 2023A Survey on the Robustness of Computer Vision Models against Common Corruptionscited by 20
  9. 2022Revisiting adapters with adversarial trainingcited by 18
  10. 2021Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselinescited by 17
  11. 2022Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priorscited by 17
  12. 2022Adversarial Robustness for Tabular Data through Cost and Utility Awarenesscited by 14
38 more
  1. 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generationcited by 14
  2. 2021Combining Diverse Feature Priorscited by 12
  3. 2022Improving Robustness by Enhancing Weak Subnetscited by 12
  4. 2023HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustnesscited by 12
  5. 2022If a Human Can See It, So Should Your System: Reliability Requirements for Machine Vision Componentscited by 11
  6. 2022Part-Based Models Improve Adversarial Robustnesscited by 11
  7. 2021Understanding Generalization in Adversarial Training via the Bias-Variance Decompositioncited by 10
  8. 2021Interpolated Joint Space Adversarial Training for Robust and Generalizable Defensescited by 10
  9. 2023Benchmarking Robustness to Adversarial Image Obfuscationscited by 8
  10. 2024Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptationcited by 8
  11. 2024The Pitfalls and Promise of Conformal Inference Under Adversarial Attackscited by 8
  12. 2022Joint rotational invariance and adversarial training of a dual-stream Transformer yields state of the art Brain-Score for Area V4cited by 7
  13. 2022A Survey on AI Sustainability: Emerging Trends on Learning Algorithms and Research Challengescited by 7
  14. 2023Benchmark Generation Framework with Customizable Distortions for Image Classifier Robustnesscited by 7
  15. 2023Adversarially Robust Spiking Neural Networks Through Conversioncited by 7
  16. 2021Semantic Perturbations with Normalizing Flows for Improved Generalizationcited by 6
  17. 2022How Robust are Discriminatively Trained Zero-Shot Learning Models?cited by 6
  18. 2023AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentationcited by 6
  19. 2024Adversarial Training: A Surveycited by 6
  20. 2022Improved post-hoc probability calibration for out-of-domain MRI segmentationcited by 5
  21. 2025Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clientscited by 5
  22. 2025Adopting a human developmental visual diet yields robust, shape-based AI visioncited by 5
  23. 2022On Certifying and Improving Generalization to Unseen Domainscited by 4
  24. 2023Investigating the Corruption Robustness of Image Classifiers with Random Lp-norm Corruptionscited by 4
  25. 2023Neural Image Compression: Generalization, Robustness, and Spectral Biasescited by 4
  26. 2023OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shiftcited by 4
  27. 2021Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Trainingcited by 3
  28. 2022Empirical Advocacy of Bio-inspired Models for Robust Image Recognitioncited by 3
  29. 2022Towards Alternative Techniques for Improving Adversarial Robustness: Analysis of Adversarial Training at a Spectrum of Perturbationscited by 3
  30. 2023Model Monitoring and Robustness of In-Use Machine Learning Models: Quantifying Data Distribution Shifts Using Population Stability Indexcited by 3
  31. 2023PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusioncited by 3
  32. 2023On the Fly Neural Style Smoothing for Risk-Averse Domain Generalizationcited by 3
  33. 2023RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Trainingcited by 3
  34. 2025Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generationcited by 3
  35. 2022Catastrophic overfitting can be induced with discriminative non-robust featurescited by 2
  36. 2022Provably Adversarially Robust Nearest Prototype Classifierscited by 2
  37. 2024Enhancing Adversarial Robustness via Uncertainty-Aware Distributional Adversarial Trainingcited by 2
  38. 2024Pretrained Reversible Generation as Unsupervised Visual Representation Learningcited by 2

50 most cited of 71 All 71 (JSON)