# NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search

arXiv: https://arxiv.org/abs/2001.10422 · HTML: https://mlsanity.com/paper/2001.10422/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2020.

## Built on by (most cited first, 30 of 64)

- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO](https://mlsanity.com/paper/2109.06716/) (2021)
- [How Powerful are Performance Predictors in Neural Architecture Search?](https://mlsanity.com/paper/2104.01177/) (2021)
- [β-DARTS: Beta-Decay Regularization for Differentiable Architecture Search](https://mlsanity.com/paper/2203.01665/) (2022)
- [BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2103.12424/) (2021)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [AMLB: an AutoML Benchmark](https://mlsanity.com/paper/2207.12560/) (2022)
- [iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients](https://mlsanity.com/paper/2106.10784/) (2021)
- [Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization](https://mlsanity.com/paper/2210.01628/) (2022)
- [NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy](https://mlsanity.com/paper/2201.13396/) (2022)
- [Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?](https://mlsanity.com/paper/2301.12844/) (2023)
- [NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks](https://mlsanity.com/paper/2110.05668/) (2021)
- [NASI: Label- and Data-agnostic Neural Architecture Search at Initialization](https://mlsanity.com/paper/2109.00817/) (2021)
- [Towards Green Automated Machine Learning: Status Quo and Future Directions](https://mlsanity.com/paper/2111.05850/) (2021)
- [NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search](https://mlsanity.com/paper/2206.09166/) (2022)
- [EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search](https://mlsanity.com/paper/2102.08099/) (2021)
- [Analyzing and Mitigating Interference in Neural Architecture Search](https://mlsanity.com/paper/2108.12821/) (2021)
- [NAS-Bench-x11 and the Power of Learning Curves](https://mlsanity.com/paper/2111.03602/) (2021)
- [Rethinking Neural Operations for Diverse Tasks](https://mlsanity.com/paper/2103.15798/) (2021)
- [Accelerating Neural Architecture Search via Proxy Data](https://mlsanity.com/paper/2106.04784/) (2021)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [EC-NAS: Energy Consumption Aware Tabular Benchmarks for Neural Architecture Search](https://mlsanity.com/paper/2210.06015/) (2022)
- [FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer](https://mlsanity.com/paper/2403.12821/) (2024)
- [HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models](https://mlsanity.com/paper/2405.10299/) (2024)
- [Zero-Cost Operation Scoring in Differentiable Architecture Search](https://mlsanity.com/paper/2106.06799/) (2021)
- [EEEA-Net: An Early Exit Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2108.06156/) (2021)
- [NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameters](https://mlsanity.com/paper/2110.10165/) (2021)
- [Neural Architecture Search as Program Transformation Exploration](https://mlsanity.com/paper/2102.06599/) (2021)
- [Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search](https://mlsanity.com/paper/2104.05309/) (2021)

## Abstract (arXiv)

One-shot neural architecture search (NAS) has played a crucial role in making NAS methods computationally feasible in practice. Nevertheless, there is still a lack of understanding on how these weight-sharing algorithms exactly work due to the many factors controlling the dynamics of the process. In order to allow a scientific study of these components, we introduce a general framework for one-shot NAS that can be instantiated to many recently-introduced variants and introduce a general benchmarking framework that draws on the recent large-scale tabular benchmark NAS-Bench-101 for cheap anytime evaluations of one-shot NAS methods. To showcase the framework, we compare several state-of-the-art one-shot NAS methods, examine how sensitive they are to their hyperparameters and how they can be improved by tuning their hyperparameters, and compare their performance to that of blackbox optimizers for NAS-Bench-101.