mlsanityData to 7 Oct 2026

RobArch: Designing Robust Architectures against Adversarial Attacks

arXiv 2301.03110 ↗First posted 8 Jan 2023cs.CV cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Adversarial Training is the most effective approach for improving the robustness of Deep Neural Networks (DNNs). However, compared to the large body of research in optimizing the adversarial training process, there are few investigations into how architecture components affect robustness, and they rarely constrain model capacity. Thus, it is unclear where robustness precisely comes from. In this work, we present the first large-scale systematic study on the robustness of DNN architecture components under fixed parameter budgets. Through our investigation, we distill 18 actionable robust network design guidelines that empower model developers to gain deep insights. We demonstrate these guidelines' effectiveness by introducing the novel Robust Architecture (RobArch) model that instantiates the guidelines to build a family of top-performing models across parameter capacities against strong adversarial attacks. RobArch achieves the new state-of-the-art AutoAttack accuracy on the RobustBench ImageNet leaderboard. The code is available at hrefhttps://github.com/ShengYun-Peng/RobArchthis url.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 58 of its 64 references that are in the map. First posted on arXiv 8 Jan 2023.

Builds on

58 of 64 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  9. 2013Intriguing properties of neural networkscited by 5,114
  10. 2017Squeeze-and-Excitation Networkscited by 4,763
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  12. 2022A ConvNet for the 2020scited by 3,545
38 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  6. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  7. 2017Mixed Precision Trainingcited by 1,135
  8. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  9. 2020Designing Network Design Spacescited by 954
  10. 2016Adversarial Machine Learning at Scalecited by 934
  11. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  12. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  13. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
  14. 2017Adversarial Patchcited by 514
  15. 2022Diffusion Models for Adversarial Purificationcited by 469
  16. 2017Mitigating Adversarial Effects Through Randomizationcited by 382
  17. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  18. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  19. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  20. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  21. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 243
  22. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  23. 2021Are Transformers More Robust Than CNNs?cited by 187
  24. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
  25. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  26. 2018DPatch: An Adversarial Patch Attack on Object Detectorscited by 146
  27. 2020Physically Realizable Adversarial Examples for LiDAR Object Detectioncited by 137
  28. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  29. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  30. 2021On the Adversarial Robustness of Vision Transformerscited by 104
  31. 2020Smooth Adversarial Trainingcited by 89
  32. 2019When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attackscited by 87
  33. 2020Adversarial Distributional Training for Robust Deep Learningcited by 79
  34. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
  35. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
  36. 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
  37. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  38. 2022A Light Recipe to Train Robust Vision Transformerscited by 64

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 2.

6 papers.

  1. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  2. 2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
  3. 2024Revisiting Adversarial Training at Scalecited by 33
  4. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  5. 2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustnesscited by 12
  6. 2024Adversarial Training: A Surveycited by 6