mlsanityData to 7 Oct 2026

Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition

arXiv 2102.01063 ↗First posted 1 Feb 2021cs.CV cs.LGcited by 71

Not on the map. Why?

Abstract (arXiv)

Accuracy predictor is a key component in Neural Architecture Search (NAS) for ranking architectures. Building a high-quality accuracy predictor usually costs enormous computation. To address this issue, instead of using an accuracy predictor, we propose a novel zero-shot index dubbed Zen-Score to rank the architectures. The Zen-Score represents the network expressivity and positively correlates with the model accuracy. The calculation of Zen-Score only takes a few forward inferences through a randomly initialized network, without training network parameters. Built upon the Zen-Score, we further propose a new NAS algorithm, termed as Zen-NAS, by maximizing the Zen-Score of the target network under given inference budgets. Within less than half GPU day, Zen-NAS is able to directly search high performance architectures in a data-free style. Comparing with previous NAS methods, the proposed Zen-NAS is magnitude times faster on multiple server-side and mobile-side GPU platforms with state-of-the-art accuracy on ImageNet. Our source code and pre-trained models are released on https://github.com/idstcv/ZenNAS.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 60 of its 69 references that are in the map. First posted on arXiv 1 Feb 2021.

Builds on

60 of 69 references.

References · 60

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  11. 2017Random Erasing Data Augmentationcited by 1,235
  12. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
38 more
  1. 2020Designing Network Design Spacescited by 954
  2. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  3. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  4. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  5. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  6. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  7. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  8. 2017Progressive Neural Architecture Searchcited by 483
  9. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  10. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  11. 2020ResNeSt: Split-Attention Networkscited by 445
  12. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  13. 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
  14. 2020Meta Pseudo Labelscited by 372
  15. 2017Large-Scale Evolution of Image Classifierscited by 348
  16. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  17. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  18. 2016On the Expressive Power of Deep Neural Networkscited by 302
  19. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  20. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  21. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  22. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  23. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  24. 2020Neural Architecture Search without Trainingcited by 241
  25. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  26. 2021Zero-Cost Proxies for Lightweight NAScited by 194
  27. 2019Complexity of Linear Regions in Deep Networkscited by 166
  28. 2018Neural Architecture Optimizationcited by 164
  29. 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
  30. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  31. 2016Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivitycited by 138
  32. 2017Genetic CNNcited by 124
  33. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  34. 2019Neural Predictor for Neural Architecture Searchcited by 116
  35. 2019Exploring Randomly Wired Neural Networks for Image Recognitioncited by 103
  36. 2019Knowledge Distillation from Internal Representationscited by 87
  37. 2016Why Deep Neural Networks for Function Approximation?cited by 82
  38. 2019Blockwisely Supervised Neural Architecture Search with Knowledge Distillationcited by 81

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 19; 2024: 19; 2025: 13; 2026: 12.

71 papers.

Sort:

By area:

  1. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  2. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
  3. 2022NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxiescited by 36
  4. 2023DisWOT: Student Architecture Search for Distillation WithOut Trainingcited by 29
  5. 2024AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Searchcited by 25
  6. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
  7. 2023Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunitiescited by 21
  8. 2021GradSign: Model Performance Inference with Theoretical Insightscited by 18
  9. 2022Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?cited by 18
  10. 2022Prior-Guided One-shot Neural Architecture Searchcited by 12
  11. 2024SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAScited by 11
  12. 2024Surprisingly Strong Performance Prediction with Neural Graph Featurescited by 11
38 more
  1. 2025Enhancing LLM-Based Neural Network Generation: Few-Shot Prompting and Efficient Validation for Automated Architecture Designcited by 10
  2. 2026From Memorization to Creativity: LLM as a Designer of Novel Neural Architecturescited by 10
  3. 2022Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammarscited by 9
  4. 2023RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxiescited by 7
  5. 2024LPZero: Language Model Zero-cost Proxy Search from Zerocited by 7
  6. 2026Resource-Efficient Iterative LLM-Based NAS with Feedback Memorycited by 7
  7. 2023Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Searchcited by 6
  8. 2023AutoST: Training-free Neural Architecture Search for Spiking Transformerscited by 6
  9. 2023Maximizing Spatio-Temporal Entropy of Deep 3D CNNs for Efficient Video Recognitioncited by 5
  10. 2023YOLOBench: Benchmarking Efficient Object Detectors on Embedded Systemscited by 5
  11. 2024ParZC: Parametric Zero-Cost Proxies for Efficient NAScited by 5
  12. 2024CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Frameworkcited by 5
  13. 2025L-SWAG: Layer-Sample Wise Activation with Gradients information for Zero-Shot NAS on Vision Transformerscited by 5
  14. 2024Robustifying and Boosting Training-Free Neural Architecture Searchcited by 4
  15. 2024Efficient Training of Large Vision Models via Advanced Automated Progressive Learningcited by 4
  16. 2025Subnet-Aware Dynamic Supernet Training for Neural Architecture Searchcited by 4
  17. 2023Efficient Model Adaptation for Continual Learning at the Edgecited by 3
  18. 2024Optimizing Time Series Forecasting Architectures: A Hierarchical Neural Architecture Search Approachcited by 3
  19. 2021A Data-driven Approach to Neural Architecture Search Initializationcited by 2
  20. 2023Entropic Score metric: Decoupling Topology and Size in Training-free NAScited by 2
  21. 2024Differentiable Model Scaling using Differentiable Topkcited by 2
  22. 2024A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanismcited by 2
  23. 2025Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weightscited by 2
  24. 2025ZeroLM: Data-Free Transformer Architecture Search for Language Modelscited by 2
  25. 2025PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptationcited by 2
  26. 2022Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Surveycited by 1
  27. 2023Neural Architecture Search: Two Constant Shared Weights Initialisationscited by 1
  28. 2023An Evaluation of Zero-Cost Proxies -- from Neural Architecture Performance to Model Robustnesscited by 1
  29. 2023Brain-inspired Evolutionary Architectures for Spiking Neural Networkscited by 1
  30. 2023ZiCo-BC: A Bias Corrected Zero-Shot NAS for Vision Taskscited by 1
  31. 2023Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisonscited by 1
  32. 2023SiGeo: Sub-One-Shot NAS via Information Theory and Geometry of Loss Landscapecited by 1
  33. 2024SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Searchcited by 1
  34. 2024NASH: Neural Architecture and Accelerator Search for Multiplication-Reduced Hybrid Modelscited by 1
  35. 2024GreenMachine: Automatic Design of Zero-Cost Proxies for Energy-Efficient NAScited by 1
  36. 2025GreenFactory: Ensembling Zero-Cost Proxies to Estimate Performance of Neural Networkscited by 1
  37. 2025Zero-Shot Neural Architecture Search with Weighted Response Correlationcited by 1
  38. 2025Dextr: Zero-Shot Neural Architecture Search with Singular Value Decomposition and Extrinsic Curvaturecited by 1

50 most cited of 71 All 71 (JSON)