mlsanityData to 7 Oct 2026

A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions

arXiv 2006.02903 ↗First posted 1 Jun 2020cs.LG stat.MLcited by 253

Not on the map. Why?

Abstract (arXiv)

Deep learning has made breakthroughs and substantial in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers' prior knowledge and experience. And due to the limitations of human' inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search (NAS) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. Besides, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.

Key facts. Cited by 253 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 47; 2022: 34; 2023: 38; 2024: 68; 2025: 44; 2026: 22.

253 papers.

Sort:

By area:

  1. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  2. 2025Multi-agent Architecture Search via Agentic Supernetcited by 164
  3. 2022Minimizing the Accumulated Trajectory Error to Improve Dataset Distillationcited by 142
  4. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  5. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  6. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  7. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 89
  8. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 84
  9. 2021Dynamic Slimmable Networkcited by 83
  10. 2022Source-Free Unsupervised Domain Adaptation: A Surveycited by 79
  11. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  12. 2024NetLLM: Adapting Large Language Models for Networkingcited by 76
38 more
  1. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 71
  2. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  3. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  4. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
  5. 2024A Comprehensive Survey on Graph Reduction: Sparsification, Coarsening, and Condensationcited by 57
  6. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  7. 2023AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuningcited by 49
  8. 2023Sequential Subset Matching for Dataset Distillationcited by 40
  9. 2022Hardware Approximate Techniques for Deep Neural Network Accelerators: A Surveycited by 39
  10. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  11. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  12. 2024KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Searchcited by 36
  13. 2024Continual Forgetting for Pre-trained Vision Modelscited by 29
  14. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  15. 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 26
  16. 2023Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectivescited by 24
  17. 2023Green Edge AI: A Contemporary Surveycited by 24
  18. 2024YOLO advances to its genesis: a decadal and comprehensive review of the You Only Look Once (YOLO) seriescited by 22
  19. 2024What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?cited by 22
  20. 2022Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problemscited by 21
  21. 2023Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunitiescited by 21
  22. 2023TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognitioncited by 21
  23. 2021Modality specific U-Net variants for biomedical image segmentation: A surveycited by 20
  24. 2022Single Model Uncertainty Estimation via Stochastic Data Centeringcited by 20
  25. 2023Deep Learning for Inertial Positioning: A Surveycited by 20
  26. 2024A CLIP-Powered Framework for Robust and Generalizable Data Selectioncited by 20
  27. 2021AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Taskscited by 19
  28. 2021Active Learning for Deep Visual Trackingcited by 19
  29. 2024Graph Data Condensation via Self-expressive Graph Structure Reconstructioncited by 18
  30. 2022Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessmentcited by 17
  31. 2022Pruning Networks with Cross-Layer Ranking & k-Reciprocal Nearest Filterscited by 16
  32. 2023Label-Efficient Learning in Agriculture: A Comprehensive Reviewcited by 16
  33. 2022Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cellscited by 15
  34. 2023Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Modelscited by 15
  35. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 15
  36. 2022Generalization Properties of NAS under Activation and Skip Connection Searchcited by 14
  37. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  38. 2023Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AIcited by 12

50 most cited of 253 All 253 (JSON)