# Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty Search

arXiv: https://arxiv.org/abs/2407.20656 · HTML: https://mlsanity.com/paper/2407.20656/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [SNIP: Single-shot Network Pruning based on Connection Sensitivity](https://mlsanity.com/paper/1810.02340/) (2018)
- [Pruning neural networks without any data by iteratively conserving synaptic flow](https://mlsanity.com/paper/2006.05467/) (2020)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [Neural Architecture Search without Training](https://mlsanity.com/paper/2006.04647/) (2020)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [Zero-Cost Proxies for Lightweight NAS](https://mlsanity.com/paper/2101.08134/) (2021)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search](https://mlsanity.com/paper/2007.10396/) (2020)
- [Evaluating Efficient Performance Estimators of Neural Architectures](https://mlsanity.com/paper/2008.03064/) (2020)
- [NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies](https://mlsanity.com/paper/2210.03230/) (2022)
- [Evolving Neural Architecture Using One Shot Model](https://mlsanity.com/paper/2012.12540/) (2020)
- [PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2204.12726/) (2022)
- [Evolution of Swarm Robotics Systems with Novelty Search](https://mlsanity.com/paper/1304.3362/) (2013)
- [Neural Architecture Search using Progressive Evolution](https://mlsanity.com/paper/2203.01559/) (2022)
- [Novelty Driven Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2204.00188/) (2022)

## Abstract (arXiv)

Neural Architecture Search (NAS) aims to automate the discovery of high-performing deep neural network architectures. Traditional objective-based NAS approaches typically optimize a certain performance metric (e.g., prediction accuracy), overlooking large parts of the architecture search space that potentially contain interesting network configurations. Furthermore, objective-driven population-based metaheuristics in complex search spaces often quickly exhaust population diversity and succumb to premature convergence to local optima. This issue becomes more complicated in NAS when performance objectives do not fully align with the actual performance of the candidate architectures, as is often the case with training-free metrics. While training-free metrics have gained popularity for their rapid performance estimation of candidate architectures without incurring computation-heavy network training, their effective incorporation into NAS remains a challenge. This paper presents the Pareto Dominance-based Novelty Search for multi-objective NAS with Multiple Training-Free metrics (MTF-PDNS). Unlike conventional NAS methods that optimize explicit objectives, MTF-PDNS promotes population diversity by utilizing a novelty score calculated based on multiple training-free performance and complexity metrics, thereby yielding a broader exploration of the search space. Experimental results on standard NAS benchmark suites demonstrate that MTF-PDNS outperforms conventional methods driven by explicit objectives in terms of convergence speed, diversity maintenance, architecture transferability, and computational costs.