# One-Shot Neural Architecture Search via Self-Evaluated Template Network

arXiv: https://arxiv.org/abs/1910.05733 · HTML: https://mlsanity.com/paper/1910.05733/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2019.

## Built on by (most cited first, 30 of 69)

- [iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients](https://mlsanity.com/paper/2106.10784/) (2021)
- [PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation](https://mlsanity.com/paper/2108.10710/) (2021)
- [Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets](https://mlsanity.com/paper/2107.00860/) (2021)
- [Training-free Transformer Architecture Search](https://mlsanity.com/paper/2203.12217/) (2022)
- [NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks](https://mlsanity.com/paper/2110.05668/) (2021)
- [DisWOT: Student Architecture Search for Distillation WithOut Training](https://mlsanity.com/paper/2303.15678/) (2023)
- [EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search](https://mlsanity.com/paper/2102.08099/) (2021)
- [DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Models](https://mlsanity.com/paper/2305.16943/) (2023)
- [Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator](https://mlsanity.com/paper/2103.07289/) (2021)
- [Learning Where To Look -- Generative NAS is Surprisingly Efficient](https://mlsanity.com/paper/2203.08734/) (2022)
- [Generic Neural Architecture Search via Regression](https://mlsanity.com/paper/2108.01899/) (2021)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Prior-Guided One-shot Neural Architecture Search](https://mlsanity.com/paper/2206.13329/) (2022)
- [A Benchmark Study on Calibration](https://mlsanity.com/paper/2308.11838/) (2023)
- [SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS](https://mlsanity.com/paper/2403.04161/) (2024)
- [Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation](https://mlsanity.com/paper/2108.04541/) (2021)
- [Reconsidering CO2 emissions from Computer Vision](https://mlsanity.com/paper/2104.08702/) (2021)
- [Less is More: Proxy Datasets in NAS approaches](https://mlsanity.com/paper/2203.06905/) (2022)
- [RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search](https://mlsanity.com/paper/2109.05691/) (2021)
- [PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2204.12726/) (2022)
- [Harmonic-NAS: Hardware-Aware Multimodal Neural Architecture Search on Resource-constrained Devices](https://mlsanity.com/paper/2309.06612/) (2023)
- [NADER: Neural Architecture Design via Multi-Agent Collaboration](https://mlsanity.com/paper/2412.19206/) (2024)
- [Neural Ensemble Search via Bayesian Sampling](https://mlsanity.com/paper/2109.02533/) (2021)
- [GP-NAS-ensemble: a model for NAS Performance Prediction](https://mlsanity.com/paper/2301.09231/) (2023)
- [MixtureGrowth: Growing Neural Networks by Recombining Learned Parameters](https://mlsanity.com/paper/2311.04251/) (2023)
- [SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Models](https://mlsanity.com/paper/2502.20422/) (2025)
- [Approximate Neural Architecture Search via Operation Distribution Learning](https://mlsanity.com/paper/2111.04670/) (2021)
- [EGANS: Evolutionary Generative Adversarial Network Search for Zero-Shot Learning](https://mlsanity.com/paper/2308.09915/) (2023)
- [BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule](https://mlsanity.com/paper/2111.13204/) (2021)
- [Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search](https://mlsanity.com/paper/2201.12725/) (2022)

## Abstract (arXiv)

Neural architecture search (NAS) aims to automate the search procedure of architecture instead of manual design. Even if recent NAS approaches finish the search within days, lengthy training is still required for a specific architecture candidate to get the parameters for its accurate evaluation. Recently one-shot NAS methods are proposed to largely squeeze the tedious training process by sharing parameters across candidates. In this way, the parameters for each candidate can be directly extracted from the shared parameters instead of training them from scratch. However, they have no sense of which candidate will perform better until evaluation so that the candidates to evaluate are randomly sampled and the top-1 candidate is considered the best. In this paper, we propose a Self-Evaluated Template Network (SETN) to improve the quality of the architecture candidates for evaluation so that it is more likely to cover competitive candidates. SETN consists of two components: (1) an evaluator, which learns to indicate the probability of each individual architecture being likely to have a lower validation loss. The candidates for evaluation can thus be selectively sampled according to this evaluator. (2) a template network, which shares parameters among all candidates to amortize the training cost of generated candidates. In experiments, the architecture found by SETN achieves state-of-the-art performance on CIFAR and ImageNet benchmarks within comparable computation costs. Code is publicly available on GitHub: https://github.com/D-X-Y/AutoDL-Projects.