mlsanityData to 14 Oct 2026

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

arXiv 2110.03825 ↗First posted 7 Oct 2021cs.LG cs.CV stat.MLcited by 74

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. A range of defense methods have been proposed to train adversarially robust DNNs, among which adversarial training has demonstrated promising results. However, despite preliminary understandings developed for adversarial training, it is still not clear, from the architectural perspective, what configurations can lead to more robust DNNs. In this paper, we address this gap via a comprehensive investigation on the impact of network width and depth on the robustness of adversarially trained DNNs. Specifically, we make the following key observations: 1) more parameters (higher model capacity) does not necessarily help adversarial robustness; 2) reducing capacity at the last stage (the last group of blocks) of the network can actually improve adversarial robustness; and 3) under the same parameter budget, there exists an optimal architectural configuration for adversarial robustness. We also provide a theoretical analysis explaning why such network configuration can help robustness. These architectural insights can help design adversarially robust DNNs. Code is available at urlhttps://github.com/HanxunH/RobustWRN.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 68 of its 96 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

68 of 96 references.

References · 68

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  10. 2016Wide Residual Networkscited by 3,070
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  12. 2018DARTS: Differentiable Architecture Searchcited by 1,798
38 more
  1. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  2. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  3. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  4. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  5. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  6. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  7. 2019Adversarial Training for Free!cited by 643
  8. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  9. 2017Countering Adversarial Images using Input Transformationscited by 520
  10. 2020Overfitting in adversarially robust deep learningcited by 506
  11. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  12. 2017Mitigating Adversarial Effects Through Randomizationcited by 383
  13. 2018Adversarially Robust Generalization Requires More Datacited by 318
  14. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 316
  15. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 314
  16. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  17. 2019Adversarial Examples Improve Image Recognitioncited by 286
  18. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 282
  19. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
  20. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
  21. 2020Bag of Tricks for Adversarial Trainingcited by 200
  22. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  23. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  24. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 183
  25. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 172
  26. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  27. 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
  28. 2019Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systemscited by 153
  29. 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 153
  30. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
  31. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  32. 2020Adversarial Camouflage: Hiding Physical-World Attacks with Natural Stylescited by 126
  33. 2019Adversarially Robust Distillationcited by 125
  34. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  35. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  36. 2019When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attackscited by 87
  37. 2020Adversarial Distributional Training for Robust Deep Learningcited by 79
  38. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 24; 2023: 18; 2024: 16; 2025: 9; 2026: 7.

74 papers.

Sort:

By area:

  1. 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
  2. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  3. 2023Robust Evaluation of Diffusion-Based Adversarial Purificationcited by 65
  4. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  5. 2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
  6. 2022Self-Ensemble Adversarial Training for Improved Robustnesscited by 38
  7. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 34
  8. 2022Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspectivecited by 28
  9. 2022Robust Models are less Over-Confidentcited by 25
  10. 2023Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noisecited by 25
  11. 2022Exploring the Relationship between Architecture and Adversarially Robust Generalizationcited by 23
  12. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
38 more
  1. 2022CNN Filter DB: An Empirical Investigation of Trained Convolutional Filterscited by 20
  2. 2022Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)cited by 18
  3. 2023TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalizationcited by 18
  4. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  5. 2023Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernelscited by 13
  6. 2023Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?cited by 13
  7. 2022Understanding Robust Learning through the Lens of Representation Similaritiescited by 12
  8. 2022Diversified Adversarial Attacks based on Conjugate Gradient Methodcited by 11
  9. 2022Enhancing Adversarial Robustness for Deep Metric Learningcited by 10
  10. 2022Squeeze Training for Adversarial Robustnesscited by 10
  11. 2023Benign Overfitting in Deep Neural Networks under Lazy Trainingcited by 10
  12. 2022Adversarial Robustness through the Lens of Convolutional Filterscited by 9
  13. 2022How many perturbations break this model? Evaluating robustness beyond adversarial accuracycited by 9
  14. 2024The Surprising Harmfulness of Benign Overfitting for Adversarial Robustnesscited by 9
  15. 2023Understanding the robustness difference between stochastic gradient descent and adaptive gradient methodscited by 8
  16. 2023MultiRobustBench: Benchmarking Robustness Against Multiple Attackscited by 7
  17. 2023RobArch: Designing Robust Architectures against Adversarial Attackscited by 6
  18. 2024Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Trainingcited by 6
  19. 2024Robust NAS under adversarial training: benchmark, theory, and beyondcited by 6
  20. 2022Boosting Adversarial Robustness From The Perspective of Effective Margin Regularizationcited by 4
  21. 2024Dense Hopfield Networks in the Teacher-Student Settingcited by 4
  22. 2022Towards Alternative Techniques for Improving Adversarial Robustness: Analysis of Adversarial Training at a Spectrum of Perturbationscited by 3
  23. 2022Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architecturescited by 3
  24. 2023RBFormer: Improve Adversarial Robustness of Transformer by Robust Biascited by 3
  25. 2024Exploring the Adversarial Frontier: Quantifying Robustness via Adversarial Hypervolumecited by 3
  26. 2024Towards Accurate and Robust Architectures via Neural Architecture Searchcited by 3
  27. 2022Alternating Objectives Generates Stronger PGD-Based Adversarial Attackscited by 2
  28. 2022Differentiable Search of Accurate and Robust Architecturescited by 2
  29. 2023RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillationcited by 2
  30. 2023Rethinking PGD Attack: Is Sign Function Necessary?cited by 2
  31. 2024Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysiscited by 2
  32. 2022Chaos Theory and Adversarial Robustnesscited by 1
  33. 2023Group-based Robustness: A General Framework for Customized Robustness in the Real Worldcited by 1
  34. 2023IRAD: Implicit Representation-driven Image Resampling against Adversarial Attackscited by 1
  35. 2024Sparse and Transferable Universal Singular Vectors Attackcited by 1
  36. 2024Investigating and unmasking feature-level vulnerabilities of CNNs to adversarial perturbationscited by 1
  37. 2024Exploring Layerwise Adversarial Robustness Through the Lens of t-SNEcited by 1
  38. 2024First line of defense: A robust first layer mitigates adversarial attackscited by 1

50 most cited of 74 All 74 (JSON)