mlsanityData to 7 Oct 2026

Data Augmentation Can Improve Robustness

arXiv 2111.05328 ↗First posted 9 Nov 2021cs.CV cs.LG stat.MLcited by 193

Not on the map. Why?

Abstract (arXiv)

Adversarial training suffers from robust overfitting, a phenomenon where the robust test accuracy starts to decrease during training. In this paper, we focus on reducing robust overfitting by using common data augmentation schemes. We demonstrate that, contrary to previous findings, when combined with model weight averaging, data augmentation can significantly boost robust accuracy. Furthermore, we compare various augmentations techniques and observe that spatial composition techniques work the best for adversarial training. Finally, we evaluate our approach on CIFAR-10 against ℓ_∞ and ℓ2 norm-bounded perturbations of size ε = 8/255 and ε = 128/255, respectively. We show large absolute improvements of +2.93% and +2.16% in robust accuracy compared to previous state-of-the-art methods. In particular, against ℓ_∞ norm-bounded perturbations of size ε = 8/255, our model reaches 60.07% robust accuracy without using any external data. We also achieve a significant performance boost with this approach while using other architectures and datasets such as CIFAR-100, SVHN and TinyImageNet.

Key facts. Cited by 193 papers in the map (data through 7 Oct 2026). Builds on 48 of its 72 references that are in the map. First posted on arXiv 9 Nov 2021.

Builds on

48 of 72 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  9. 2016Wide Residual Networkscited by 3,057
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  11. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  12. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
36 more
  1. 2016Adversarial examples in the physical worldcited by 1,762
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  6. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  7. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  8. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  9. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  10. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  11. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  12. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  13. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  14. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  15. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  16. 2020Overfitting in adversarially robust deep learningcited by 504
  17. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  18. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  19. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  20. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  21. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  22. 2018Adversarial Logit Pairingcited by 224
  23. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  24. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  25. 2019Are Labels Required for Improving Adversarial Robustness?cited by 162
  26. 2019Adversarial Robustness through Local Linearizationcited by 146
  27. 2019Privacy Risks of Securing Machine Learning Models against Adversarial Examplescited by 136
  28. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  29. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  30. 2020Learnable Boundary Guided Adversarial Trainingcited by 93
  31. 2019Adversarially Robust Generalization Just Requires More Unlabeled Datacited by 75
  32. 2019Robustness to Adversarial Perturbations in Learning from Incomplete Datacited by 49
  33. 2019An Alternative Surrogate Loss for PGD-based Adversarial Testingcited by 45
  34. 2017Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networkscited by 31
  35. 2018Logit Pairing Methods Can Fool Gradient-Based Attackscited by 27
  36. 2018Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensemblescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 34; 2023: 59; 2024: 55; 2025: 35; 2026: 9.

193 papers.

Sort:

By area:

  1. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  2. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  3. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  4. 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
  5. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  6. 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
  7. 2024Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefitscited by 65
  8. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  9. 2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
  10. 2023Data Augmentation Alone Can Improve Adversarial Trainingcited by 43
  11. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  12. 2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
38 more
  1. 2023Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasingcited by 31
  2. 2023Robust Mixture-of-Expert Training for Convolutional Neural Networkscited by 30
  3. 2022StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translationcited by 28
  4. 2022Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspectivecited by 28
  5. 2022A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspectivecited by 27
  6. 2024ADBM: Adversarial diffusion bridge model for reliable adversarial purificationcited by 24
  7. 2022Exploring the Relationship between Architecture and Adversarially Robust Generalizationcited by 23
  8. 2023Seasoning Model Soups for Robustness to Adversarial and Natural Distribution Shiftscited by 23
  9. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systemscited by 23
  10. 2022Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognitioncited by 22
  11. 2024An End-to-End Robust Point Cloud Semantic Segmentation Network with Single-Step Conditional Diffusion Modelscited by 22
  12. 2022A Comprehensive Survey on Data-Efficient GANs in Image Generationcited by 20
  13. 2022Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attackscited by 18
  14. 2022Revisiting adapters with adversarial trainingcited by 18
  15. 2022Recognizing Object by Components with Human Prior Knowledge Enhances Adversarial Robustness of Deep Neural Networkscited by 18
  16. 2023TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalizationcited by 18
  17. 2023Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspectivecited by 18
  18. 2023Toward Degree Bias in Embedding-Based Knowledge Graph Completioncited by 17
  19. 2025Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learningcited by 17
  20. 2025Security of Internet of Agents: Attacks and Countermeasurescited by 17
  21. 2024Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuningcited by 15
  22. 2024NLP Verification: Towards a General Methodology for Certifying Robustnesscited by 14
  23. 2023Black-box Targeted Adversarial Attack on Segment Anything (SAM)cited by 13
  24. 2024Transformers Learn Low Sensitivity Functions: Investigations and Implicationscited by 13
  25. 2024Revisiting Adversarial Training under Long-Tailed Distributionscited by 13
  26. 2022Increasing Confidence in Adversarial Robustness Evaluationscited by 12
  27. 2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustnesscited by 12
  28. 2022Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samplescited by 11
  29. 2023Angler: Helping Machine Translation Practitioners Prioritize Model Improvementscited by 11
  30. 2023Mitigating Adversarial Vulnerability through Causal Parameter Estimation by Adversarial Double Machine Learningcited by 11
  31. 2024DAFA: Distance-Aware Fair Adversarial Trainingcited by 11
  32. 2024Constrained Adaptive Attack: Effective Adversarial Attack Against Deep Neural Networks for Tabular Datacited by 11
  33. 2024PhysAug: A Physical-guided and Frequency-based Data Augmentation for Single-Domain Generalized Object Detectioncited by 11
  34. 2023Advising OpenMP Parallelization via a Graph-Based Approach with Transformerscited by 10
  35. 2023An Extended Study of Human-like Behavior under Adversarial Trainingcited by 9
  36. 2023Visual Data-Type Understanding does not emerge from Scaling Vision-Language Modelscited by 9
  37. 2025Fast Adversarial Training with Weak-to-Strong Spatial-Temporal Consistency in the Frequency Domain on Videoscited by 9
  38. 2022GeoECG: Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Predictioncited by 8

50 most cited of 193 All 193 (JSON)