mlsanityData to 7 Oct 2026

Robust Principles: Architectural Design Principles for Adversarially Robust CNNs

arXiv 2308.16258 ↗First posted 30 Aug 2023cs.CVcited by 46

Not on the map. Why?

Abstract (arXiv)

Our research aims to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs. To accomplish our goal, we synthesize a suite of three generalizable robust architectural design principles: (a) optimal range for depth and width configurations, (b) preferring convolutional over patchify stem stage, and (c) robust residual block design through adopting squeeze and excitation blocks and non-parametric smooth activation functions. Through extensive experiments across a wide spectrum of dataset scales, adversarial training methods, model parameters, and network design spaces, our principles consistently and markedly improve AutoAttack accuracy: 1-3 percentage points (pp) on CIFAR-10 and CIFAR-100, and 4-9 pp on ImageNet. The code is publicly available at https://github.com/poloclub/robust-principles.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 47 of its 57 references that are in the map. First posted on arXiv 30 Aug 2023.

Builds on

47 of 57 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2016Wide Residual Networkscited by 3,057
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
35 more
  1. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  2. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  3. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  4. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  5. 2017Mixed Precision Trainingcited by 1,135
  6. 2020Designing Network Design Spacescited by 954
  7. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  8. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  9. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  10. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  11. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
  12. 2022Diffusion Models for Adversarial Purificationcited by 469
  13. 2021Early Convolutions Help Transformers See Bettercited by 395
  14. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  15. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  16. 2021Are Transformers More Robust Than CNNs?cited by 187
  17. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  18. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  19. 2017L2 Regularization versus Batch and Weight Normalizationcited by 121
  20. 2020Smooth Adversarial Trainingcited by 89
  21. 2019When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attackscited by 87
  22. 2020Adversarial Distributional Training for Robust Deep Learningcited by 79
  23. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  24. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
  25. 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
  26. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  27. 2022A Light Recipe to Train Robust Vision Transformerscited by 64
  28. 2022When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecturecited by 64
  29. 2021Rapid Neural Architecture Search by Learning to Generate Graphs from Datasetscited by 33
  30. 2021Vision Transformers with Hierarchical Attentioncited by 31
  31. 2021AdvRush: Searching for Adversarially Robust Neural Architecturescited by 28
  32. 2022Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspectivecited by 28
  33. 2021Parameterizing Activation Functions for Adversarial Robustnesscited by 22
  34. 2023Neural Architecture Design and Robustness: A Datasetcited by 15
  35. 2023RobArch: Designing Robust Architectures against Adversarial Attackscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 25; 2025: 8; 2026: 8.

46 papers.

Sort:

By area:

  1. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  2. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  3. 2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
  4. 2024Securely Fine-tuning Pre-trained Encoders Against Adversarial Examplescited by 21
  5. 2024MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsificationcited by 20
  6. 2024MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifierscited by 16
  7. 2025One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIPcited by 15
  8. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  9. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  10. 2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustnesscited by 12
  11. 2024Ensemble everything everywhere: Multi-scale aggregation for adversarial robustnesscited by 12
  12. 2025Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Schedulercited by 12
34 more
  1. 2024Shedding More Light on Robust Classifiers under the lens of Energy-based Modelscited by 9
  2. 2024Adversarial Guided Diffusion Models for Adversarial Purificationcited by 8
  3. 2024σ-zero: Gradient-based Optimization of ℓ0-norm Adversarial Examplescited by 7
  4. 2023Assessing Robustness via Score-Based Adversarial Image Generationcited by 6
  5. 2023High-Performance Transformers for Table Structure Recognition Need Early Convolutionscited by 4
  6. 2024A Random Ensemble of Encrypted Vision Transformers for Adversarially Robust Defensecited by 3
  7. 2024A Notion of Uniqueness for the Adversarial Bayes Classifiercited by 3
  8. 2024Data-Driven Lipschitz Continuity: A Cost-Effective Approach to Improve Adversarial Robustnesscited by 3
  9. 2024Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attackscited by 3
  10. 2025GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy Algorithmcited by 3
  11. 2024Large-Scale Dataset Pruning in Adversarial Training through Data Importance Extrapolationcited by 2
  12. 2024Criticality Leveraged Adversarial Training (CLAT) for Boosted Performance via Parameter Efficiencycited by 2
  13. 2024On the Robustness of Adversarial Training Against Uncertainty Attackscited by 2
  14. 2024RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defensescited by 2
  15. 2025Understanding Adversarial Training with Energy-based Modelscited by 2
  16. 2023IRAD: Implicit Representation-driven Image Resampling against Adversarial Attackscited by 1
  17. 2024Adversarial Examples are Misaligned in Diffusion Model Manifoldscited by 1
  18. 2024RAMP: Boosting Adversarial Robustness Against Multiple lp Perturbations for Universal Robustnesscited by 1
  19. 2024Robust width: A lightweight and certifiable adversarial defensecited by 1
  20. 2024Towards Generalized Certified Robustness with Multi-Norm Trainingcited by 1
  21. 2026OTI: A Model-free and Visually Interpretable Measure of Image Attackabilitycited by 1
  22. 2026Solving adversarial examples requires solving exponential misalignmentcited by 1
  23. 2024On adversarial training and the 1 Nearest Neighbor classifiercited by 0
  24. 2024Enhancing Output Diversity Improves Conjugate Gradient-based Adversarial Attackscited by 0
  25. 2024Top-GAP: Integrating Size Priors in CNNs for more Interpretability, Robustness, and Bias Mitigationcited by 0
  26. 2025Model-Free Adversarial Purification via Coarse-To-Fine Tensor Network Representationcited by 0
  27. 2025Seeking Flat Minima over Diverse Surrogates for Improved Adversarial Transferability: A Theoretical Framework and Algorithmic Instantiationcited by 0
  28. 2025Dynamic Parameter Optimization for Highly Transferable Transformation-Based Attackscited by 0
  29. 2026Expanding the Role of Diffusion Models for Robust Classifier Trainingcited by 0
  30. 2026Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerabilitycited by 0
  31. 2026NERO-Net: A Neuroevolutionary Approach for the Design of Adversarially Robust CNNscited by 0
  32. 2026FastAT Benchmark: A Comprehensive Framework for Fair Evaluation of Fast Adversarial Training Methodscited by 0
  33. 2026The Scissors Effect: When Resize-Based Input Diversity Helps or Hurts Transfer Attackscited by 0
  34. 2026VanillaBench: The Hidden Accuracy Cost of Adversarial Robustnesscited by 0