mlsanityData to 14 Oct 2026

Neural Architecture Search with Random Labels

arXiv 2101.11834 ↗First posted 28 Jan 2021cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate a new variant of neural architecture search (NAS) paradigm -- searching with random labels (RLNAS). The task sounds counter-intuitive for most existing NAS algorithms since random label provides few information on the performance of each candidate architecture. Instead, we propose a novel NAS framework based on ease-of-convergence hypothesis, which requires only random labels during searching. The algorithm involves two steps: first, we train a SuperNet using random labels; second, from the SuperNet we extract the sub-network whose weights change most significantly during the training. Extensive experiments are evaluated on multiple datasets (e.g. NAS-Bench-201 and ImageNet) and multiple search spaces (e.g. DARTS-like and MobileNet-like). Very surprisingly, RLNAS achieves comparable or even better results compared with state-of-the-art NAS methods such as PC-DARTS, Single Path One-Shot, even though the counterparts utilize full ground truth labels for searching. We hope our finding could inspire new understandings on the essential of NAS.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 39 of its 50 references that are in the map. First posted on arXiv 28 Jan 2021.

Builds on

39 of 50 references.

References · 39

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  9. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
27 more
  1. 2016Colorful Image Colorizationcited by 1,114
  2. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  3. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  4. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  5. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  6. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  7. 2017Progressive Neural Architecture Searchcited by 487
  8. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  9. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  10. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  11. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 291
  12. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
  13. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  14. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  15. 2020Neural Architecture Search without Trainingcited by 244
  16. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  17. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 202
  18. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  19. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 130
  20. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  21. 2019DetNAS: Backbone Search for Object Detectioncited by 81
  22. 2020Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?cited by 59
  23. 2020What Do Neural Networks Learn When Trained With Random Labels?cited by 59
  24. 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
  25. 2020Are Labels Necessary for Neural Architecture Search?cited by 37
  26. 2020Angle-based Search Space Shrinking for Neural Architecture Searchcited by 32
  27. 2018BlockQNN: Efficient Block-wise Neural Network Architecture Generationcited by 31

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 13; 2023: 5; 2024: 4; 2025: 3.

33 papers.

Sort:

  1. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  2. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 58
  3. 2022Neural Architecture Search for Spiking Neural Networkscited by 53
  4. 2022AutoSNN: Towards Energy-Efficient Spiking Neural Networkscited by 40
  5. 2024AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Searchcited by 28
  6. 2021Analyzing and Mitigating Interference in Neural Architecture Searchcited by 20
  7. 2021Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shiftcited by 14
  8. 2022The Curious Case of Benign Memorizationcited by 12
  9. 2021Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metricscited by 7
  10. 2023β-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Searchcited by 7
  11. 2023Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformerscited by 7
  12. 2021Observations on K-image Expansion of Image-Mixing Augmentation for Classificationcited by 5
21 more
  1. 2024Unsupervised Graph Neural Architecture Search with Disentangled Self-supervisioncited by 5
  2. 2022Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysiscited by 4
  3. 2022Differentiable Architecture Search with Random Featurescited by 4
  4. 2024DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisionscited by 4
  5. 2024OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strengthcited by 4
  6. 2022When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Searchcited by 3
  7. 2022Automated Dominative Subspace Mining for Efficient Neural Architecture Searchcited by 3
  8. 2022Revisiting Training-free NAS Metrics: An Efficient Training-based Methodcited by 3
  9. 2023GeNAS: Neural Architecture Search with Better Generalizationcited by 3
  10. 2025Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weightscited by 3
  11. 2021Training BatchNorm Only in Neural Architecture Search and Beyondcited by 2
  12. 2022Towards Self-supervised and Weight-preserving Neural Architecture Searchcited by 2
  13. 2022Leveraging Unlabeled Data to Track Memorizationcited by 2
  14. 2022One Network Doesn't Rule Them All: Moving Beyond Handcrafted Architectures in Self-Supervised Learningcited by 1
  15. 2022α DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modelingcited by 1
  16. 2025Dextr: Zero-Shot Neural Architecture Search with Singular Value Decomposition and Extrinsic Curvaturecited by 1
  17. 2021Connection Sensitivity Matters for Training-free DARTS: From Architecture-Level Scoring to Operation-Level Sensitivity Analysiscited by 0
  18. 2021Mutually-aware Sub-Graphs Differentiable Architecture Searchcited by 0
  19. 2022UnrealNAS: Can We Search Neural Architectures with Unreal Data?cited by 0
  20. 2023Robust MAE-Driven NAS: From Mask Reconstruction to Architecture Innovationcited by 0
  21. 2025Is Supervised Learning Really That Different from Unsupervised?cited by 0