# To Share or Not To Share: A Comprehensive Appraisal of Weight-Sharing

arXiv: https://arxiv.org/abs/2002.04289 · HTML: https://mlsanity.com/paper/2002.04289/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 11 of 11)

- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [Generalizing Few-Shot NAS with Gradient Matching](https://mlsanity.com/paper/2203.15207/) (2022)
- [Analyzing and Mitigating Interference in Neural Architecture Search](https://mlsanity.com/paper/2108.12821/) (2021)
- [Neural Architecture Search for Dense Prediction Tasks in Computer Vision](https://mlsanity.com/paper/2202.07242/) (2022)
- [Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"](https://mlsanity.com/paper/2302.00932/) (2023)
- [Rethink DARTS Search Space and Renovate a New Benchmark](https://mlsanity.com/paper/2306.06852/) (2023)
- [Robustifying DARTS by Eliminating Information Bypass Leakage via Explicit Sparse Regularization](https://mlsanity.com/paper/2306.06858/) (2023)
- [Towards Efficient Few-shot Graph Neural Architecture Search via Partitioning Gradient Contribution](https://mlsanity.com/paper/2506.01231/) (2025)
- [AutoHEnsGNN: Winning Solution to AutoGraph Challenge for KDD Cup 2020](https://mlsanity.com/paper/2111.12952/) (2021)
- [Learning to Rank Ace Neural Architectures via Normalized Discounted Cumulative Gain](https://mlsanity.com/paper/2108.03001/) (2021)
- [Delve into the Performance Degradation of Differentiable Architecture Search](https://mlsanity.com/paper/2109.13466/) (2021)

## Abstract (arXiv)

Weight-sharing (WS) has recently emerged as a paradigm to accelerate the automated search for efficient neural architectures, a process dubbed Neural Architecture Search (NAS). Although very appealing, this framework is not without drawbacks and several works have started to question its capabilities on small hand-crafted benchmarks. In this paper, we take advantage of the nasbench dataset to challenge the efficiency of WS on a representative search space. By comparing a SOTA WS approach to a plain random search we show that, despite decent correlations between evaluations using weight-sharing and standalone ones, WS is only rarely significantly helpful to NAS. In particular we highlight the impact of the search space itself on the benefits.