mlsanityData to 7 Oct 2026

A Survey on Computationally Efficient Neural Architecture Search

arXiv 2206.01520 ↗First posted 3 Jun 2022cs.LG cs.NEcited by 12

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) has become increasingly popular in the deep learning community recently, mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks (DNNs). However, NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS, and training DNNs is computationally intensive. To solve this major limitation of NAS, improving the computational efficiency is essential in the design of NAS. However, a systematic overview of computationally efficient NAS (CE-NAS) methods still lacks. To fill this gap, we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods, together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities. The remaining challenges and open research questions are also discussed, and promising research topics in this emerging field are suggested.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 106 of its 175 references that are in the map. First posted on arXiv 3 Jun 2022.

Builds on

106 of 175 references.

References · 106

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2017Mask R-CNNcited by 7,259
  8. 2016Densely Connected Convolutional Networkscited by 6,691
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  11. 2018How Powerful are Graph Neural Networks?cited by 4,355
  12. 2014Going Deeper with Convolutionscited by 4,231
38 more
  1. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  2. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  3. 2015Fast R-CNNcited by 2,930
  4. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  7. 2018Federated Learning with Non-IID Datacited by 1,099
  8. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  9. 2019Deep Leakage from Gradientscited by 1,004
  10. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  11. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  12. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  13. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  14. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  15. 2015Holistically-Nested Edge Detectioncited by 590
  16. 2017Progressive Neural Architecture Searchcited by 483
  17. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  18. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  19. 2019Federated Machine Learning: Concept and Applicationscited by 409
  20. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  21. 2017Large-Scale Evolution of Image Classifierscited by 348
  22. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  23. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  24. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  25. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  26. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  27. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  28. 2015Net2Net: Accelerating Learning via Knowledge Transfercited by 245
  29. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  30. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  31. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
  32. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  33. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  34. 2018Auto-Keras: An Efficient Neural Architecture Search Systemcited by 169
  35. 2018Neural Architecture Optimizationcited by 164
  36. 2017A Survey on Homomorphic Encryption Schemes: Theory and Implementationcited by 154
  37. 2018Efficient Multi-objective Neural Architecture Search via Lamarckian Evolutioncited by 151
  38. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142

50 most cited of 106 All 106 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 3; 2025: 6; 2026: 1.

12 papers.

  1. 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
  2. 2023Real-Time Semantic Segmentation: A Brief Survey & Comparative Study in Remote Sensingcited by 4
  3. 2022Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architecturescited by 3
  4. 2024A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanismcited by 2
  5. 2024SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Searchcited by 1
  6. 2025A Genetic Algorithm-Based Approach for Automated Optimization of Kolmogorov-Arnold Networks in Classification Taskscited by 1
  7. 2025One Search Fits All: Pareto-Optimal Eco-Friendly Model Selectioncited by 1
  8. 2025Onboard Optimization and Learning: A Surveycited by 1
  9. 2024MO-EMT-NAS: Multi-Objective Continuous Transfer of Architectural Knowledge Between Tasks from Different Datasetscited by 0
  10. 2025SiamNAS: Siamese Surrogate Model for Dominance Relation Prediction in Multi-objective Neural Architecture Searchcited by 0
  11. 2025TrashDet: Iterative Neural Architecture Search for Efficient Waste Detectioncited by 0
  12. 2026HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Searchcited by 0