# Are Labels Necessary for Neural Architecture Search?

arXiv: https://arxiv.org/abs/2003.12056 · HTML: https://mlsanity.com/paper/2003.12056/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2020.

## Built on by (most cited first, 30 of 37)

- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2103.12424/) (2021)
- [TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search](https://mlsanity.com/paper/2105.11871/) (2021)
- [Contrastive Multimodal Fusion with TupleInfoNCE](https://mlsanity.com/paper/2107.02575/) (2021)
- [Neural Architecture Search with Random Labels](https://mlsanity.com/paper/2101.11834/) (2021)
- [NASI: Label- and Data-agnostic Neural Architecture Search at Initialization](https://mlsanity.com/paper/2109.00817/) (2021)
- [PT4AL: Using Self-Supervised Pretext Tasks for Active Learning](https://mlsanity.com/paper/2201.07459/) (2022)
- [Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search](https://mlsanity.com/paper/2102.04700/) (2021)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [Generic Neural Architecture Search via Regression](https://mlsanity.com/paper/2108.01899/) (2021)
- [Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift](https://mlsanity.com/paper/2108.09671/) (2021)
- [Self-supervised Cross-silo Federated Neural Architecture Search](https://mlsanity.com/paper/2101.11896/) (2021)
- [Data-Free Neural Architecture Search via Recursive Label Calibration](https://mlsanity.com/paper/2112.02086/) (2021)
- [Contrastive Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2102.10557/) (2021)
- [DATA: Domain-Aware and Task-Aware Self-supervised Learning](https://mlsanity.com/paper/2203.09041/) (2022)
- [Unsupervised Graph Neural Architecture Search with Disentangled Self-supervision](https://mlsanity.com/paper/2403.05064/) (2024)
- [Towards Less Constrained Macro-Neural Architecture Search](https://mlsanity.com/paper/2203.05508/) (2022)
- [Exploring the Intersection between Neural Architecture Search and Continual Learning](https://mlsanity.com/paper/2206.05625/) (2022)
- [Differentiable Architecture Search with Random Features](https://mlsanity.com/paper/2208.08835/) (2022)
- [DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions](https://mlsanity.com/paper/2403.01326/) (2024)
- [OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength](https://mlsanity.com/paper/2409.14433/) (2024)
- [Single-DARTS: Towards Stable Architecture Search](https://mlsanity.com/paper/2108.08128/) (2021)
- [Are Neural Architecture Search Benchmarks Well Designed? A Deeper Look Into Operation Importance](https://mlsanity.com/paper/2303.16938/) (2023)
- [Towards a Robust Differentiable Architecture Search under Label Noise](https://mlsanity.com/paper/2110.12197/) (2021)
- [Towards Self-supervised and Weight-preserving Neural Architecture Search](https://mlsanity.com/paper/2206.04125/) (2022)
- [How Much Is Hidden in the NAS Benchmarks? Few-Shot Adaptation of a NAS Predictor](https://mlsanity.com/paper/2311.18451/) (2023)
- [One Network Doesn't Rule Them All: Moving Beyond Handcrafted Architectures in Self-Supervised Learning](https://mlsanity.com/paper/2203.08130/) (2022)
- [Meta-Learning of NAS for Few-shot Learning in Medical Image Applications](https://mlsanity.com/paper/2203.08951/) (2022)
- [α DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modeling](https://mlsanity.com/paper/2211.10105/) (2022)

## Abstract (arXiv)

Existing neural network architectures in computer vision -- whether designed by humans or by machines -- were typically found using both images and their associated labels. In this paper, we ask the question: can we find high-quality neural architectures using only images, but no human-annotated labels? To answer this question, we first define a new setup called Unsupervised Neural Architecture Search (UnNAS). We then conduct two sets of experiments. In sample-based experiments, we train a large number (500) of diverse architectures with either supervised or unsupervised objectives, and find that the architecture rankings produced with and without labels are highly correlated. In search-based experiments, we run a well-established NAS algorithm (DARTS) using various unsupervised objectives, and report that the architectures searched without labels can be competitive to their counterparts searched with labels. Together, these results reveal the potentially surprising finding that labels are not necessary, and the image statistics alone may be sufficient to identify good neural architectures.