# Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS

arXiv: https://arxiv.org/abs/1911.09336 · HTML: https://mlsanity.com/paper/1911.09336/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation](https://mlsanity.com/paper/2105.12971/) (2021)
- [Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2106.11890/) (2021)
- [Learning Where To Look -- Generative NAS is Surprisingly Efficient](https://mlsanity.com/paper/2203.08734/) (2022)
- [One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2111.01203/) (2021)
- [Improving Hyperparameter Optimization with Checkpointed Model Weights](https://mlsanity.com/paper/2406.18630/) (2024)
- [Architecture-Aware Learning Curve Extrapolation via Graph Ordinary Differential Equation](https://mlsanity.com/paper/2412.15554/) (2024)
- [Efficient Automatic Machine Learning via Design Graphs](https://mlsanity.com/paper/2210.12257/) (2022)
- [A Survey on Multi-Objective Neural Architecture Search](https://mlsanity.com/paper/2307.09099/) (2023)

## Abstract (arXiv)

Neural Architecture Search (NAS) has shown great potentials in finding better neural network designs. Sample-based NAS is the most reliable approach which aims at exploring the search space and evaluating the most promising architectures. However, it is computationally very costly. As a remedy, the one-shot approach has emerged as a popular technique for accelerating NAS using weight-sharing. However, due to the weight-sharing of vastly different networks, the one-shot approach is less reliable than the sample-based approach. In this work, we propose BONAS (Bayesian Optimized Neural Architecture Search), a sample-based NAS framework which is accelerated using weight-sharing to evaluate multiple related architectures simultaneously. Specifically, we apply Graph Convolutional Network predictor as a surrogate model for Bayesian Optimization to select multiple related candidate models in each iteration. We then apply weight-sharing to train multiple candidate models simultaneously. This approach not only accelerates the traditional sample-based approach significantly, but also keeps its reliability. This is because weight-sharing among related architectures are more reliable than those in the one-shot approach. Extensive experiments are conducted to verify the effectiveness of our method over many competing algorithms.