# How Does Supernet Help in Neural Architecture Search?

arXiv: https://arxiv.org/abs/2010.08219 · HTML: https://mlsanity.com/paper/2010.08219/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [Analyzing and Mitigating Interference in Neural Architecture Search](https://mlsanity.com/paper/2108.12821/) (2021)
- [Enabling NAS with Automated Super-Network Generation](https://mlsanity.com/paper/2112.10878/) (2021)
- [Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAS](https://mlsanity.com/paper/2206.05896/) (2022)
- [Learning to Rank Ace Neural Architectures via Normalized Discounted Cumulative Gain](https://mlsanity.com/paper/2108.03001/) (2021)
- [DARTS without a Validation Set: Optimizing the Marginal Likelihood](https://mlsanity.com/paper/2112.13023/) (2021)

## Abstract (arXiv)

Weight sharing, as an approach to speed up architecture performance estimation has received wide attention. Instead of training each architecture separately, weight sharing builds a supernet that assembles all the architectures as its submodels. However, there has been debate over whether the NAS process actually benefits from weight sharing, due to the gap between supernet optimization and the objective of NAS. To further understand the effect of weight sharing on NAS, we conduct a comprehensive analysis on five search spaces, including NAS-Bench-101, NAS-Bench-201, DARTS-CIFAR10, DARTS-PTB, and ProxylessNAS. We find that weight sharing works well on some search spaces but fails on others. Taking a step forward, we further identified biases accounting for such phenomenon and the capacity of weight sharing. Our work is expected to inspire future NAS researchers to better leverage the power of weight sharing.