mlsanityData to 14 Oct 2026

Progressive Neural Architecture Search

arXiv 1712.00559 ↗First posted 2 Dec 2017cs.CV cs.LG stat.MLcited by 487

Not on the map. Why?

Abstract (arXiv)

We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms. Our approach uses a sequential model-based optimization (SMBO) strategy, in which we search for structures in order of increasing complexity, while simultaneously learning a surrogate model to guide the search through structure space. Direct comparison under the same search space shows that our method is up to 5 times more efficient than the RL method of Zoph et al. (2018) in terms of number of models evaluated, and 8 times faster in terms of total compute. The structures we discover in this way achieve state of the art classification accuracies on CIFAR-10 and ImageNet.

Key facts. Cited by 487 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 175; 2022: 121; 2023: 74; 2024: 62; 2025: 37; 2026: 17.

487 papers.

Sort:

By area:

  1. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  2. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
  3. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
  4. 2021Coordinate Attention for Efficient Mobile Network Designcited by 241
  5. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 222
  6. 2021Accounting for Variance in Machine Learning Benchmarkscited by 201
  7. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  8. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  9. 2022Compute Trends Across Three Eras of Machine Learningcited by 163
  10. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 154
  11. 2021Low-Light Image and Video Enhancement Using Deep Learning: A Surveycited by 148
  12. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
38 more
  1. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  2. 2021A Survey of Modern Deep Learning based Object Detection Modelscited by 98
  3. 2022Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shiftcited by 94
  4. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  5. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  6. 2021Diverse Branch Block: Building a Convolution as an Inception-like Unitcited by 87
  7. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 80
  8. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 79
  9. 2022Deep Model Reassemblycited by 78
  10. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 73
  11. 2021TorchGeo: Deep Learning With Geospatial Datacited by 73
  12. 2022DC-BENCH: Dataset Condensation Benchmarkcited by 67
  13. 2021Model Complexity of Deep Learning: A Surveycited by 65
  14. 2022Better Uncertainty Calibration via Proper Scores for Classification and Beyondcited by 65
  15. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  16. 2021Are we ready for a new paradigm shift? A Survey on Visual Deep MLPcited by 63
  17. 2021A Comprehensive Survey on Hardware-Aware Neural Architecture Searchcited by 61
  18. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 60
  19. 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
  20. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 58
  21. 2021Meta Gradient Adversarial Attackcited by 55
  22. 2021A Survey on Green Deep Learningcited by 52
  23. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  24. 2021Evolving Reinforcement Learning Algorithmscited by 49
  25. 2021Firefly Neural Architecture Descent: a General Approach for Growing Neural Networkscited by 49
  26. 2022Online Continual Learning for Embedded Devicescited by 49
  27. 2021DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentationcited by 48
  28. 2021Fast and Accurate Model Scalingcited by 46
  29. 2023AdvDiff: Generating Unrestricted Adversarial Examples using Diffusion Modelscited by 46
  30. 2021Contrastive Multimodal Fusion with TupleInfoNCEcited by 43
  31. 2021Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)cited by 41
  32. 2021Searching the Search Space of Vision Transformercited by 41
  33. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 41
  34. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 39
  35. 2021Pano3D: A Holistic Benchmark and a Solid Baseline for 360o Depth Estimationcited by 37
  36. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 37
  37. 2022NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxiescited by 36
  38. 2021AutoDropout: Learning Dropout Patterns to Regularize Deep Networkscited by 35

50 most cited of 487 All 487 (JSON)