# Tackling Neural Architecture Search With Quality Diversity Optimization

arXiv: https://arxiv.org/abs/2208.00204 · HTML: https://mlsanity.com/paper/2208.00204/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 64 references that are in the map.
- First posted on arXiv 30 Jul 2022.

## Builds on (most cited first, 30 of 39)

- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Non-stochastic Best Arm Identification and Hyperparameter Optimization](https://mlsanity.com/paper/1502.07943/) (2015)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Hierarchical Representations for Efficient Architecture Search](https://mlsanity.com/paper/1711.00436/) (2017)
- [BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search](https://mlsanity.com/paper/1910.11858/) (2019)
- [Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution](https://mlsanity.com/paper/1804.09081/) (2018)
- [How Powerful are Performance Predictors in Neural Architecture Search?](https://mlsanity.com/paper/2104.01177/) (2021)
- [Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours](https://mlsanity.com/paper/1904.02877/) (2019)
- [μNAS: Constrained Neural Architecture Search for Microcontrollers](https://mlsanity.com/paper/2010.14246/) (2020)
- [SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers](https://mlsanity.com/paper/1905.12107/) (2019)
- [Illuminating Mario Scenes in the Latent Space of a Generative Adversarial Network](https://mlsanity.com/paper/2007.05674/) (2020)
- [DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures](https://mlsanity.com/paper/1806.08198/) (2018)
- [A Study on Encodings for Neural Architecture Search](https://mlsanity.com/paper/2007.04965/) (2020)
- [Multi-objective Asynchronous Successive Halving](https://mlsanity.com/paper/2106.12639/) (2021)
- [Exploring the Loss Landscape in Neural Architecture Search](https://mlsanity.com/paper/2005.02960/) (2020)
- [Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization](https://mlsanity.com/paper/2105.01015/) (2021)

## Built on by (most cited first, 3 of 3)

- [MUSS: Multilevel Subset Selection for Relevance and Diversity](https://mlsanity.com/paper/2503.11126/) (2025)
- [Hardware Aware Ensemble Selection for Balancing Predictive Accuracy and Cost](https://mlsanity.com/paper/2408.02280/) (2024)
- [Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning](https://mlsanity.com/paper/2606.08735/) (2026)

## Abstract (arXiv)

Neural architecture search (NAS) has been studied extensively and has grown to become a research field with substantial impact. While classical single-objective NAS searches for the architecture with the best performance, multi-objective NAS considers multiple objectives that should be optimized simultaneously, e.g., minimizing resource usage along the validation error. Although considerable progress has been made in the field of multi-objective NAS, we argue that there is some discrepancy between the actual optimization problem of practical interest and the optimization problem that multi-objective NAS tries to solve. We resolve this discrepancy by formulating the multi-objective NAS problem as a quality diversity optimization (QDO) problem and introduce three quality diversity NAS optimizers (two of them belonging to the group of multifidelity optimizers), which search for high-performing yet diverse architectures that are optimal for application-specific niches, e.g., hardware constraints. By comparing these optimizers to their multi-objective counterparts, we demonstrate that quality diversity NAS in general outperforms multi-objective NAS with respect to quality of solutions and efficiency. We further show how applications and future NAS research can thrive on QDO.