# Neural Predictor for Neural Architecture Search

arXiv: https://arxiv.org/abs/1912.00848 · HTML: https://mlsanity.com/paper/1912.00848/

## Key facts

- Cited by 118 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2019.

## Built on by (most cited first, 30 of 118)

- [Zero-Cost Proxies for Lightweight NAS](https://mlsanity.com/paper/2101.08134/) (2021)
- [Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective](https://mlsanity.com/paper/2102.11535/) (2021)
- [How Powerful are Performance Predictors in Neural Architecture Search?](https://mlsanity.com/paper/2104.01177/) (2021)
- [Parameter Prediction for Unseen Deep Architectures](https://mlsanity.com/paper/2110.13100/) (2021)
- [Deep Model Reassembly](https://mlsanity.com/paper/2210.17409/) (2022)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [CktGNN: Circuit Graph Neural Network for Electronic Design Automation](https://mlsanity.com/paper/2308.16406/) (2023)
- [An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks](https://mlsanity.com/paper/2102.10423/) (2021)
- [LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search](https://mlsanity.com/paper/2102.04040/) (2021)
- [Stronger NAS with Weaker Predictors](https://mlsanity.com/paper/2102.10490/) (2021)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [CATE: Computation-aware Neural Architecture Encoding with Transformers](https://mlsanity.com/paper/2102.07108/) (2021)
- [NAS-Bench-x11 and the Power of Learning Curves](https://mlsanity.com/paper/2111.03602/) (2021)
- [PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphs](https://mlsanity.com/paper/2203.10304/) (2022)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [PyGlove: Symbolic Programming for Automated Machine Learning](https://mlsanity.com/paper/2101.08809/) (2021)
- [GradSign: Model Performance Inference with Theoretical Insights](https://mlsanity.com/paper/2110.08616/) (2021)
- [HAO: Hardware-aware neural Architecture Optimization for Efficient Inference](https://mlsanity.com/paper/2104.12766/) (2021)
- [DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Models](https://mlsanity.com/paper/2305.16943/) (2023)
- [Learning Where To Look -- Generative NAS is Surprisingly Efficient](https://mlsanity.com/paper/2203.08734/) (2022)
- [Homogeneous Architecture Augmentation for Neural Predictor](https://mlsanity.com/paper/2107.13153/) (2021)
- [AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models](https://mlsanity.com/paper/2509.12019/) (2025)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction](https://mlsanity.com/paper/2211.08024/) (2022)
- [RankNAS: Efficient Neural Architecture Search by Pairwise Ranking](https://mlsanity.com/paper/2109.07383/) (2021)
- [Neural Architecture Search for Dense Prediction Tasks in Computer Vision](https://mlsanity.com/paper/2202.07242/) (2022)
- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [FLASH: Fast Neural Architecture Search with Hardware Optimization](https://mlsanity.com/paper/2108.00568/) (2021)
- [EC-NAS: Energy Consumption Aware Tabular Benchmarks for Neural Architecture Search](https://mlsanity.com/paper/2210.06015/) (2022)
- [SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS](https://mlsanity.com/paper/2403.04161/) (2024)

## Abstract (arXiv)

Neural Architecture Search methods are effective but often use complex algorithms to come up with the best architecture. We propose an approach with three basic steps that is conceptually much simpler. First we train N random architectures to generate N (architecture, validation accuracy) pairs and use them to train a regression model that predicts accuracy based on the architecture. Next, we use this regression model to predict the validation accuracies of a large number of random architectures. Finally, we train the top-K predicted architectures and deploy the model with the best validation result. While this approach seems simple, it is more than 20 times as sample efficient as Regularized Evolution on the NASBench-101 benchmark and can compete on ImageNet with more complex approaches based on weight sharing, such as ProxylessNAS.