# Efficient Progressive Neural Architecture Search

arXiv: https://arxiv.org/abs/1808.00391 · HTML: https://mlsanity.com/paper/1808.00391/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2018.

## Built on by (most cited first, 8 of 8)

- [Searching the Search Space of Vision Transformer](https://mlsanity.com/paper/2111.14725/) (2021)
- [IRLAS: Inverse Reinforcement Learning for Architecture Search](https://mlsanity.com/paper/1812.05285/) (2018)
- [GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models](https://mlsanity.com/paper/2211.15155/) (2022)
- [Towards Less Constrained Macro-Neural Architecture Search](https://mlsanity.com/paper/2203.05508/) (2022)
- [Towards Better Multi-modal Keyphrase Generation via Visual Entity Enhancement and Multi-granularity Image Noise Filtering](https://mlsanity.com/paper/2309.04734/) (2023)
- [Meta-Learning of NAS for Few-shot Learning in Medical Image Applications](https://mlsanity.com/paper/2203.08951/) (2022)
- [AutoSynth: Learning to Generate 3D Training Data for Object Point Cloud Registration](https://mlsanity.com/paper/2309.11170/) (2023)
- [Automatic Fused Multimodal Deep Learning for Plant Identification](https://mlsanity.com/paper/2406.01455/) (2024)

## Abstract (arXiv)

This paper addresses the difficult problem of finding an optimal neural architecture design for a given image classification task. We propose a method that aggregates two main results of the previous state-of-the-art in neural architecture search. These are, appealing to the strong sampling efficiency of a search scheme based on sequential model-based optimization (SMBO), and increasing training efficiency by sharing weights among sampled architectures. Sequential search has previously demonstrated its capabilities to find state-of-the-art neural architectures for image classification. However, its computational cost remains high, even unreachable under modest computational settings. Affording SMBO with weight-sharing alleviates this problem. On the other hand, progressive search with SMBO is inherently greedy, as it leverages a learned surrogate function to predict the validation error of neural architectures. This prediction is directly used to rank the sampled neural architectures. We propose to attenuate the greediness of the original SMBO method by relaxing the role of the surrogate function so it predicts architecture sampling probability instead. We demonstrate with experiments on the CIFAR-10 dataset that our method, denominated Efficient progressive neural architecture search (EPNAS), leads to increased search efficiency, while retaining competitiveness of found architectures.