mlsanityData to 14 Oct 2026

Neural Predictor for Neural Architecture Search

arXiv 1912.00848 ↗First posted 2 Dec 2019cs.LG stat.MLcited by 118

Not on the map. Why?

Abstract (arXiv)

Neural Architecture Search methods are effective but often use complex algorithms to come up with the best architecture. We propose an approach with three basic steps that is conceptually much simpler. First we train N random architectures to generate N (architecture, validation accuracy) pairs and use them to train a regression model that predicts accuracy based on the architecture. Next, we use this regression model to predict the validation accuracies of a large number of random architectures. Finally, we train the top-K predicted architectures and deploy the model with the best validation result. While this approach seems simple, it is more than 20 times as sample efficient as Regularized Evolution on the NASBench-101 benchmark and can compete on ImageNet with more complex approaches based on weight sharing, such as ProxylessNAS.

Key facts. Cited by 118 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 26; 2022: 31; 2023: 21; 2024: 17; 2025: 15; 2026: 8.

118 papers.

Sort:

By area:

  1. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  2. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  3. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  4. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  5. 2022Deep Model Reassemblycited by 78
  6. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 73
  7. 2023CktGNN: Circuit Graph Neural Network for Electronic Design Automationcited by 58
  8. 2021An Evaluation of Edge TPU Accelerators for Convolutional Neural Networkscited by 39
  9. 2021LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Searchcited by 28
  10. 2021Stronger NAS with Weaker Predictorscited by 28
  11. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  12. 2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
38 more
  1. 2021NAS-Bench-x11 and the Power of Learning Curvescited by 20
  2. 2022PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphscited by 20
  3. 2022Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolutioncited by 20
  4. 2021PyGlove: Symbolic Programming for Automated Machine Learningcited by 19
  5. 2021GradSign: Model Performance Inference with Theoretical Insightscited by 19
  6. 2021HAO: Hardware-aware neural Architecture Optimization for Efficient Inferencecited by 16
  7. 2023DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Modelscited by 16
  8. 2022Learning Where To Look -- Generative NAS is Surprisingly Efficientcited by 15
  9. 2021Homogeneous Architecture Augmentation for Neural Predictorcited by 14
  10. 2025AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Modelscited by 14
  11. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  12. 2022NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Predictioncited by 13
  13. 2021RankNAS: Efficient Neural Architecture Search by Pairwise Rankingcited by 12
  14. 2022Neural Architecture Search for Dense Prediction Tasks in Computer Visioncited by 12
  15. 2022A Survey on Computationally Efficient Neural Architecture Searchcited by 12
  16. 2021FLASH: Fast Neural Architecture Search with Hardware Optimizationcited by 11
  17. 2022EC-NAS: Energy Consumption Aware Tabular Benchmarks for Neural Architecture Searchcited by 11
  18. 2024SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAScited by 11
  19. 2024Surprisingly Strong Performance Prediction with Neural Graph Featurescited by 11
  20. 2025Regression Language Models for Codecited by 11
  21. 2024FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformercited by 10
  22. 2021Learning Versatile Neural Architectures by Propagating Network Codescited by 9
  23. 2021Zero-Cost Operation Scoring in Differentiable Architecture Searchcited by 9
  24. 2021RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halvingcited by 9
  25. 2022Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Searchcited by 9
  26. 2022PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Searchcited by 8
  27. 2022AIO-P: Expanding Neural Performance Predictors Beyond Image Classificationcited by 8
  28. 2024What Are Good Positional Encodings for Directed Graphs?cited by 8
  29. 2024LayerDAG: A Layerwise Autoregressive Diffusion Model for Directed Acyclic Graph Generationcited by 8
  30. 2022NASRec: Weight Sharing Neural Architecture Search for Recommender Systemscited by 7
  31. 2023GP-NAS-ensemble: a model for NAS Performance Predictioncited by 7
  32. 2023NAR-Former V2: Rethinking Transformer for Universal Neural Network Representation Learningcited by 7
  33. 2021FNAS: Uncertainty-Aware Fast Neural Architecture Searchcited by 6
  34. 2022TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasetscited by 6
  35. 2023Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Searchcited by 6
  36. 2022Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Searchcited by 5
  37. 2022GENNAPE: Towards Generalized Neural Architecture Performance Estimatorscited by 5
  38. 2024ParZC: Parametric Zero-Cost Proxies for Efficient NAScited by 5

50 most cited of 118 All 118 (JSON)