mlsanityData to 7 Oct 2026

Efficient Neural Architecture Search via Parameter Sharing

arXiv 1802.03268 ↗First posted 9 Feb 2018cs.LG cs.CL cs.CVcited by 831

Not on the map. Why?

Abstract (arXiv)

We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design. In ENAS, a controller learns to discover neural network architectures by searching for an optimal subgraph within a large computational graph. The controller is trained with policy gradient to select a subgraph that maximizes the expected reward on the validation set. Meanwhile the model corresponding to the selected subgraph is trained to minimize a canonical cross entropy loss. Thanks to parameter sharing between child models, ENAS is fast: it delivers strong empirical performances using much fewer GPU-hours than all existing automatic model design approaches, and notably, 1000x less expensive than standard Neural Architecture Search. On the Penn Treebank dataset, ENAS discovers a novel architecture that achieves a test perplexity of 55.8, establishing a new state-of-the-art among all methods without post-training processing. On the CIFAR-10 dataset, ENAS designs novel architectures that achieve a test error of 2.89%, which is on par with NASNet (Zoph et al., 2018), whose test error is 2.65%.

Key facts. Cited by 831 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 266; 2022: 198; 2023: 135; 2024: 112; 2025: 67; 2026: 52.

831 papers.

Sort:

By area:

  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  3. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  4. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  5. 2021A Field Guide to Federated Optimizationcited by 312
  6. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  7. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
  8. 2023OpenAGI: When LLM Meets Domain Expertscited by 196
  9. 2021Zero-Cost Proxies for Lightweight NAScited by 194
  10. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
  11. 2021The Values Encoded in Machine Learning Researchcited by 172
  12. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
38 more
  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  2. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  3. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  4. 2022Minimizing the Accumulated Trajectory Error to Improve Dataset Distillationcited by 142
  5. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  6. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  7. 2022Neural Prompt Searchcited by 133
  8. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  9. 2021Involution: Inverting the Inherence of Convolution for Visual Recognitioncited by 99
  10. 2021The Efficiency Misnomercited by 98
  11. 2021QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuitscited by 97
  12. 2022Deep Image Deblurring: A Surveycited by 94
  13. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  14. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  15. 2021LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Searchcited by 82
  16. 2022Pathways: Asynchronous Distributed Dataflow for MLcited by 81
  17. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  18. 2022Deep Model Reassemblycited by 78
  19. 2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 77
  20. 2024AutoFlow: Automated Workflow Generation for Large Language Model Agentscited by 76
  21. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  22. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  23. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 71
  24. 2021Directed Acyclic Graph Neural Networkscited by 70
  25. 2022DC-BENCH: Dataset Condensation Benchmarkcited by 67
  26. 2023MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Taskscited by 66
  27. 2021Model Complexity of Deep Learning: A Surveycited by 64
  28. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  29. 2021A Comprehensive Survey on Hardware-Aware Neural Architecture Searchcited by 60
  30. 2021Quantum Architecture Search via Deep Reinforcement Learningcited by 58
  31. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  32. 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
  33. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  34. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
  35. 2022Omni-Dimensional Dynamic Convolutioncited by 55
  36. 2021A Survey on Green Deep Learningcited by 52
  37. 2022Neural Architecture Search for Spiking Neural Networkscited by 52
  38. 2025MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinementcited by 52

50 most cited of 831 All 831 (JSON)