mlsanityData to 7 Oct 2026

AdvRush: Searching for Adversarially Robust Neural Architectures

arXiv 2108.01289 ↗First posted 3 Aug 2021cs.LG cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Deep neural networks continue to awe the world with their remarkable performance. Their predictions, however, are prone to be corrupted by adversarial examples that are imperceptible to humans. Current efforts to improve the robustness of neural networks against adversarial examples are focused on developing robust training methods, which update the weights of a neural network in a more robust direction. In this work, we take a step beyond training of the weight parameters and consider the problem of designing an adversarially robust neural architecture with high intrinsic robustness. We propose AdvRush, a novel adversarial robustness-aware neural architecture search algorithm, based upon a finding that independent of the training method, the intrinsic robustness of a neural network can be represented with the smoothness of its input loss landscape. Through a regularizer that favors a candidate architecture with a smoother input loss landscape, AdvRush successfully discovers an adversarially robust neural architecture. Along with a comprehensive theoretical motivation for AdvRush, we conduct an extensive amount of experiments to demonstrate the efficacy of AdvRush on various benchmark datasets. Notably, on CIFAR-10, AdvRush achieves 55.91% robust accuracy under FGSM attack after standard training and 50.04% robust accuracy under AutoAttack after 7-step PGD adversarial training.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 67 of its 80 references that are in the map. First posted on arXiv 3 Aug 2021.

Builds on

67 of 80 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  8. 2016Adversarial examples in the physical worldcited by 1,762
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  12. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
38 more
  1. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  2. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  5. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  7. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  8. 2018Robustness May Be at Odds with Accuracycited by 873
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  11. 2016Universal adversarial perturbationscited by 774
  12. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  13. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  14. 2017One pixel attack for fooling deep neural networkscited by 579
  15. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  16. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
  17. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 459
  18. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  19. 2017Large-Scale Evolution of Image Classifierscited by 348
  20. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  21. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  22. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  23. 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 268
  24. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  25. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  26. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  27. 2018Generating Adversarial Examples with Adversarial Networkscited by 256
  28. 2019Simple Black-box Adversarial Attackscited by 249
  29. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 243
  30. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  31. 2017On Detecting Adversarial Perturbationscited by 232
  32. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  33. 2018Adversarial Logit Pairingcited by 224
  34. 2014Qualitatively characterizing neural network optimization problemscited by 186
  35. 2018Robustness via curvature regularization, and vice versacited by 174
  36. 2019Adversarial Robustness through Local Linearizationcited by 146
  37. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  38. 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 10; 2024: 8; 2025: 2; 2026: 2.

28 papers.

Sort:

By area:

  1. 2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
  2. 20213D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detectioncited by 43
  3. 2022Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspectivecited by 28
  4. 2023Neural Architecture Design and Robustness: A Datasetcited by 15
  5. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  6. 2023HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustnesscited by 12
  7. 2024HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Modelscited by 10
  8. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 8
  9. 2023Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbationscited by 6
  10. 2024Robust NAS under adversarial training: benchmark, theory, and beyondcited by 6
  11. 2023Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Surveycited by 5
  12. 2024FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustnesscited by 5
16 more
  1. 20233D Adversarial Augmentations for Robust Out-of-Domain Predictionscited by 4
  2. 2022Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architecturescited by 3
  3. 2024Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defensecited by 3
  4. 2024Towards Accurate and Robust Architectures via Neural Architecture Searchcited by 3
  5. 2022Differentiable Search of Accurate and Robust Architecturescited by 2
  6. 2023RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillationcited by 2
  7. 2023Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluationcited by 2
  8. 2023An Evaluation of Zero-Cost Proxies -- from Neural Architecture Performance to Model Robustnesscited by 1
  9. 2024Investigating and unmasking feature-level vulnerabilities of CNNs to adversarial perturbationscited by 1
  10. 2025SEAL: Searching Expandable Architectures for Incremental Learningcited by 1
  11. 2022Adversarially Robust Medical Classification via Attentive Convolutional Neural Networkscited by 0
  12. 2024Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustnesscited by 0
  13. 2024Double Oracle Neural Architecture Search for Game Theoretic Deep Learning Modelscited by 0
  14. 2025ProARD: progressive adversarial robustness distillation: provide wide range of robust studentscited by 0
  15. 2026NERO-Net: A Neuroevolutionary Approach for the Design of Adversarially Robust CNNscited by 0
  16. 2026HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Searchcited by 0