# AdvRush: Searching for Adversarially Robust Neural Architectures

arXiv: https://arxiv.org/abs/2108.01289 · HTML: https://mlsanity.com/paper/2108.01289/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 67 of its 80 references that are in the map.
- First posted on arXiv 3 Aug 2021.

## Builds on (most cited first, 30 of 67)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models](https://mlsanity.com/paper/1805.06605/) (2018)

## Built on by (most cited first, 28 of 28)

- [Robust Principles: Architectural Design Principles for Adversarially Robust CNNs](https://mlsanity.com/paper/2308.16258/) (2023)
- [3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection](https://mlsanity.com/paper/2112.04764/) (2021)
- [Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspective](https://mlsanity.com/paper/2212.11005/) (2022)
- [Neural Architecture Design and Robustness: A Dataset](https://mlsanity.com/paper/2306.06712/) (2023)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustness](https://mlsanity.com/paper/2307.11823/) (2023)
- [HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models](https://mlsanity.com/paper/2405.10299/) (2024)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations](https://mlsanity.com/paper/2306.05031/) (2023)
- [Robust NAS under adversarial training: benchmark, theory, and beyond](https://mlsanity.com/paper/2403.13134/) (2024)
- [Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey](https://mlsanity.com/paper/2303.06302/) (2023)
- [FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustness](https://mlsanity.com/paper/2402.19102/) (2024)
- [3D Adversarial Augmentations for Robust Out-of-Domain Predictions](https://mlsanity.com/paper/2308.15479/) (2023)
- [Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architectures](https://mlsanity.com/paper/2207.05321/) (2022)
- [Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defense](https://mlsanity.com/paper/2402.18787/) (2024)
- [Towards Accurate and Robust Architectures via Neural Architecture Search](https://mlsanity.com/paper/2405.05502/) (2024)
- [Differentiable Search of Accurate and Robust Architectures](https://mlsanity.com/paper/2212.14049/) (2022)
- [RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation](https://mlsanity.com/paper/2301.08092/) (2023)
- [Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation](https://mlsanity.com/paper/2305.07308/) (2023)
- [An Evaluation of Zero-Cost Proxies -- from Neural Architecture Performance to Model Robustness](https://mlsanity.com/paper/2307.09365/) (2023)
- [Investigating and unmasking feature-level vulnerabilities of CNNs to adversarial perturbations](https://mlsanity.com/paper/2405.20672/) (2024)
- [SEAL: Searching Expandable Architectures for Incremental Learning](https://mlsanity.com/paper/2505.10457/) (2025)
- [Adversarially Robust Medical Classification via Attentive Convolutional Neural Networks](https://mlsanity.com/paper/2210.14405/) (2022)
- [Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustness](https://mlsanity.com/paper/2406.06792/) (2024)
- [Double Oracle Neural Architecture Search for Game Theoretic Deep Learning Models](https://mlsanity.com/paper/2410.04764/) (2024)
- [ProARD: progressive adversarial robustness distillation: provide wide range of robust students](https://mlsanity.com/paper/2506.07666/) (2025)
- [NERO-Net: A Neuroevolutionary Approach for the Design of Adversarially Robust CNNs](https://mlsanity.com/paper/2603.25517/) (2026)
- [HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Search](https://mlsanity.com/paper/2605.04103/) (2026)

## Abstract (arXiv)

Deep neural networks continue to awe the world with their remarkable performance. Their predictions, however, are prone to be corrupted by adversarial examples that are imperceptible to humans. Current efforts to improve the robustness of neural networks against adversarial examples are focused on developing robust training methods, which update the weights of a neural network in a more robust direction. In this work, we take a step beyond training of the weight parameters and consider the problem of designing an adversarially robust neural architecture with high intrinsic robustness. We propose AdvRush, a novel adversarial robustness-aware neural architecture search algorithm, based upon a finding that independent of the training method, the intrinsic robustness of a neural network can be represented with the smoothness of its input loss landscape. Through a regularizer that favors a candidate architecture with a smoother input loss landscape, AdvRush successfully discovers an adversarially robust neural architecture. Along with a comprehensive theoretical motivation for AdvRush, we conduct an extensive amount of experiments to demonstrate the efficacy of AdvRush on various benchmark datasets. Notably, on CIFAR-10, AdvRush achieves 55.91% robust accuracy under FGSM attack after standard training and 50.04% robust accuracy under AutoAttack after 7-step PGD adversarial training.