mlsanityData to 7 Oct 2026

Evaluating the Search Phase of Neural Architecture Search

arXiv 1902.08142 ↗First posted 21 Feb 2019cs.LG stat.MLcited by 120

Not on the map. Why?

Abstract (arXiv)

Neural Architecture Search (NAS) aims to facilitate the design of deep networks for new tasks. Existing techniques rely on two stages: searching over the architecture space and validating the best architecture. NAS algorithms are currently compared solely based on their results on the downstream task. While intuitive, this fails to explicitly evaluate the effectiveness of their search strategies. In this paper, we propose to evaluate the NAS search phase. To this end, we compare the quality of the solutions obtained by NAS search policies with that of random architecture selection. We find that: (i) On average, the state-of-the-art NAS algorithms perform similarly to the random policy; (ii) the widely-used weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We believe that our evaluation framework will be key to designing NAS strategies that consistently discover architectures superior to random ones.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 49; 2022: 22; 2023: 17; 2024: 17; 2025: 6; 2026: 9.

120 papers.

Sort:

By area:

  1. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  2. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  3. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  4. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  5. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  6. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  7. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 71
  8. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  9. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
  10. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  11. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  12. 2022NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easycited by 36
38 more
  1. 2021Neural Architecture Search with Random Labelscited by 33
  2. 2021Contrastive Neural Architecture Search with Neural Architecture Comparatorscited by 33
  3. 2021Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Searchcited by 27
  4. 2022Generalizing Few-Shot NAS with Gradient Matchingcited by 25
  5. 2021Core-set Sampling for Efficient Neural Architecture Searchcited by 23
  6. 2022Shapley-NAS: Discovering Operation Contribution for Neural Architecture Searchcited by 23
  7. 2023LEMON: Lossless model expansioncited by 23
  8. 2022Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learningcited by 22
  9. 2023Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunitiescited by 21
  10. 2021Analyzing and Mitigating Interference in Neural Architecture Searchcited by 20
  11. 2021ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxiescited by 20
  12. 2021NAS-Bench-x11 and the Power of Learning Curvescited by 20
  13. 2021ZARTS: On Zero-order Optimization for Neural Architecture Searchcited by 17
  14. 2021Masked Training of Neural Networks with Partial Gradientscited by 15
  15. 2021Generic Neural Architecture Search via Regressioncited by 14
  16. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  17. 2022Neural Architecture Search for Dense Prediction Tasks in Computer Visioncited by 11
  18. 2023ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devicescited by 10
  19. 2021Learning Versatile Neural Architectures by Propagating Network Codescited by 9
  20. 2021Generative Adversarial Neural Architecture Searchcited by 9
  21. 2021Zero-Cost Operation Scoring in Differentiable Architecture Searchcited by 9
  22. 2021Neural Architecture Search as Program Transformation Explorationcited by 8
  23. 2021Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Searchcited by 8
  24. 2021Multi-objective Optimization by Learning Space Partitionscited by 8
  25. 2022PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Searchcited by 8
  26. 2023Enabling Resource-efficient AIoT System with Cross-level Optimization: A surveycited by 8
  27. 2021Fisher Task Distance and Its Application in Neural Architecture Searchcited by 7
  28. 2021Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metricscited by 7
  29. 2022GPUNet: Searching the Deployable Convolution Neural Networks for GPUscited by 7
  30. 2026Resource-Efficient Iterative LLM-Based NAS with Feedback Memorycited by 7
  31. 2021Adversarial Branch Architecture Search for Unsupervised Domain Adaptationcited by 6
  32. 2021Neighborhood-Aware Neural Architecture Searchcited by 6
  33. 2021L2NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learningcited by 6
  34. 2021Approximate Neural Architecture Search via Operation Distribution Learningcited by 6
  35. 2023PA&DA: Jointly Sampling PAth and DAta for Consistent NAScited by 6
  36. 2021Contrastive Self-supervised Neural Architecture Searchcited by 5
  37. 2021AutoSpace: Neural Architecture Search with Less Human Interferencecited by 5
  38. 2021BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rulecited by 5

50 most cited of 120 All 120 (JSON)