mlsanityData to 7 Oct 2026

Fixing Data Augmentation to Improve Adversarial Robustness

arXiv 2103.01946 ↗First posted 2 Mar 2021cs.CV cs.LGcited by 201

Not on the map. Why?

Cited by 23 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Adversarial training suffers from robust overfitting, a phenomenon where the robust test accuracy starts to decrease during training. In this paper, we focus on both heuristics-driven and data-driven augmentations as a means to reduce robust overfitting. First, we demonstrate that, contrary to previous findings, when combined with model weight averaging, data augmentation can significantly boost robust accuracy. Second, we explore how state-of-the-art generative models can be leveraged to artificially increase the size of the training set and further improve adversarial robustness. Finally, we evaluate our approach on CIFAR-10 against ℓ_∞ and ℓ2 norm-bounded perturbations of size ε = 8/255 and ε = 128/255, respectively. We show large absolute improvements of +7.06% and +5.88% in robust accuracy compared to previous state-of-the-art methods. In particular, against ℓ_∞ norm-bounded perturbations of size ε = 8/255, our model reaches 64.20% robust accuracy without using any external data, beating most prior works that use external data.

Key facts. Cited by 201 papers in the map (data through 7 Oct 2026). Builds on 60 of its 74 references that are in the map. First posted on arXiv 2 Mar 2021.

Builds on

60 of 74 references.

References · 60

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2016Wide Residual Networkscited by 3,057
  12. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
38 more
  1. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  2. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  3. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  4. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  5. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  6. 2016Adversarial examples in the physical worldcited by 1,762
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  8. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  12. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  13. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  14. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  15. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  16. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  17. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  18. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  19. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  20. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  21. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  22. 2020Overfitting in adversarially robust deep learningcited by 504
  23. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  24. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  25. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  26. 2018Adversarially Robust Generalization Requires More Datacited by 318
  27. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  28. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  29. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  30. 2019On the "steerability" of generative adversarial networkscited by 247
  31. 2018Adversarial Logit Pairingcited by 224
  32. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  33. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  34. 2019Are Labels Required for Improving Adversarial Robustness?cited by 162
  35. 2019Adversarial Robustness through Local Linearizationcited by 146
  36. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  37. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  38. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 56; 2023: 58; 2024: 37; 2025: 25; 2026: 8.

201 papers.

Sort:

By area:

  1. 2022Diffusion Models for Adversarial Purificationcited by 469
  2. 2021Unsolved Problems in ML Safetycited by 291
  3. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  4. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  5. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  6. 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
  7. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
  8. 2023Robust Classification via a Single Diffusion Modelcited by 79
  9. 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
  10. 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 65
  11. 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
  12. 2021HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noisecited by 61
38 more
  1. 2023X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detectioncited by 58
  2. 2023Decoupled Kullback-Leibler Divergence Losscited by 58
  3. 2021Score-Based Generative Classifierscited by 53
  4. 2021Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Themcited by 46
  5. 2022Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentationscited by 46
  6. 2022Practical Evaluation of Adversarial Robustness via Adaptive Auto Attackcited by 44
  7. 2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
  8. 2021Consistency Regularization for Adversarial Robustnesscited by 42
  9. 2022Self-Ensemble Adversarial Training for Improved Robustnesscited by 38
  10. 2023Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labelscited by 37
  11. 2022Probabilistically Robust Learning: Balancing Average- and Worst-case Performancecited by 36
  12. 2023DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purificationcited by 36
  13. 2023Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustnesscited by 35
  14. 2023Improving Fast Adversarial Training with Prior-Guided Knowledgecited by 34
  15. 2023DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protectioncited by 34
  16. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 33
  17. 2022Stability Analysis and Generalization Bounds of Adversarial Trainingcited by 30
  18. 2022FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfittingcited by 28
  19. 2023Randomized Adversarial Training via Taylor Expansioncited by 28
  20. 2021Noisy Feature Mixupcited by 27
  21. 2022Robust Models are less Over-Confidentcited by 25
  22. 2023Toward Understanding Generative Data Augmentationcited by 25
  23. 2024Your Diffusion Model is Secretly a Certifiably Robust Classifiercited by 24
  24. 2023Improving Generalization of Adversarial Training via Robust Critical Fine-Tuningcited by 23
  25. 2021Parameterizing Activation Functions for Adversarial Robustnesscited by 22
  26. 2022The Enemy of My Enemy is My Friend: Exploring Inverse Adversaries for Improving Adversarial Trainingcited by 22
  27. 2022CNN Filter DB: An Empirical Investigation of Trained Convolutional Filterscited by 20
  28. 2022Scaling Adversarial Training to Large Perturbation Boundscited by 20
  29. 2023DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothingcited by 20
  30. 2023Enhancing Adversarial Robustness via Score-Based Optimizationcited by 19
  31. 2022Adversarial Rademacher Complexity of Deep Neural Networkscited by 18
  32. 2023Improving the Accuracy-Robustness Trade-Off of Classifiers via Adaptive Smoothingcited by 18
  33. 2023MimicDiffusion: Purifying Adversarial Perturbation via Mimicking Clean Diffusion Modelcited by 18
  34. 2021Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselinescited by 17
  35. 2022NoisyMix: Boosting Model Robustness to Common Corruptionscited by 17
  36. 2022On the benefits of knowledge distillation for adversarial robustnesscited by 17
  37. 2022Improving Robustness against Real-World and Worst-Case Distribution Shifts through Decision Region Quantificationcited by 16
  38. 2022Robust Trajectory Prediction against Adversarial Attackscited by 16

50 most cited of 201 All 201 (JSON)