# Effective, Efficient and Robust Neural Architecture Search

arXiv: https://arxiv.org/abs/2011.09820 · HTML: https://mlsanity.com/paper/2011.09820/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 11 of 11)

- [Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond](https://mlsanity.com/paper/2501.10945/) (2025)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Learning Diverse-Structured Networks for Adversarial Robustness](https://mlsanity.com/paper/2102.01886/) (2021)
- [A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization](https://mlsanity.com/paper/2401.09257/) (2024)
- [Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architectures](https://mlsanity.com/paper/2207.05321/) (2022)
- [Evolutionary Neural Architecture Search for Transformer in Knowledge Tracing](https://mlsanity.com/paper/2310.01180/) (2023)
- [DASS: Differentiable Architecture Search for Sparse neural networks](https://mlsanity.com/paper/2207.06968/) (2022)
- [RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation](https://mlsanity.com/paper/2301.08092/) (2023)
- [A Survey on Multi-Objective Neural Architecture Search](https://mlsanity.com/paper/2307.09099/) (2023)
- [ProARD: progressive adversarial robustness distillation: provide wide range of robust students](https://mlsanity.com/paper/2506.07666/) (2025)
- [Proactive Disentangled Modeling of Trigger-Object Pairings for Backdoor Defense](https://mlsanity.com/paper/2508.01932/) (2025)

## Abstract (arXiv)

Recent advances in adversarial attacks show the vulnerability of deep neural networks searched by Neural Architecture Search (NAS). Although NAS methods can find network architectures with the state-of-the-art performance, the adversarial robustness and resource constraint are often ignored in NAS. To solve this problem, we propose an Effective, Efficient, and Robust Neural Architecture Search (E2RNAS) method to search a neural network architecture by taking the performance, robustness, and resource constraint into consideration. The objective function of the proposed E2RNAS method is formulated as a bi-level multi-objective optimization problem with the upper-level problem as a multi-objective optimization problem, which is different from existing NAS methods. To solve the proposed objective function, we integrate the multiple-gradient descent algorithm, a widely studied gradient-based multi-objective optimization algorithm, with the bi-level optimization. Experiments on benchmark datasets show that the proposed E2RNAS method can find adversarially robust architectures with optimized model size and comparable classification accuracy.