# EZNAS: Evolving Zero Cost Proxies For Neural Architecture Scoring

arXiv: https://arxiv.org/abs/2209.07413 · HTML: https://mlsanity.com/paper/2209.07413/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 25 of 25)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [Neural Architecture Search without Training](https://mlsanity.com/paper/2006.04647/) (2020)
- [Zero-Cost Proxies for Lightweight NAS](https://mlsanity.com/paper/2101.08134/) (2021)
- [Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective](https://mlsanity.com/paper/2102.11535/) (2021)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)
- [NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size](https://mlsanity.com/paper/2009.00437/) (2020)
- [Neuro-Symbolic Program Synthesis](https://mlsanity.com/paper/1611.01855/) (2016)
- [CARS: Continuous Evolution for Efficient Neural Architecture Search](https://mlsanity.com/paper/1909.04977/) (2019)
- [DSNAS: Direct Neural Architecture Search without Parameter Retraining](https://mlsanity.com/paper/2002.09128/) (2020)
- [On Network Design Spaces for Visual Recognition](https://mlsanity.com/paper/1905.13214/) (2019)
- [EcoNAS: Finding Proxies for Economical Neural Architecture Search](https://mlsanity.com/paper/2001.01233/) (2020)
- [A Study on Encodings for Neural Architecture Search](https://mlsanity.com/paper/2007.04965/) (2020)
- [GradSign: Model Performance Inference with Theoretical Insights](https://mlsanity.com/paper/2110.08616/) (2021)
- [DANCE: Differentiable Accelerator/Network Co-Exploration](https://mlsanity.com/paper/2009.06237/) (2020)
- [DNA: Differentiable Network-Accelerator Co-Search](https://mlsanity.com/paper/2010.14778/) (2020)
- [Enabling NAS with Automated Super-Network Generation](https://mlsanity.com/paper/2112.10878/) (2021)
- [RHNAS: Realizable Hardware and Neural Architecture Search](https://mlsanity.com/paper/2106.09180/) (2021)

## Built on by (most cited first, 15 of 15)

- [Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models](https://mlsanity.com/paper/2406.02924/) (2024)
- [Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery](https://mlsanity.com/paper/2312.09059/) (2023)
- [LPZero: Language Model Zero-cost Proxy Search from Zero](https://mlsanity.com/paper/2410.04808/) (2024)
- [Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Search](https://mlsanity.com/paper/2306.02459/) (2023)
- [ParZC: Parametric Zero-Cost Proxies for Efficient NAS](https://mlsanity.com/paper/2402.02105/) (2024)
- [Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envision](https://mlsanity.com/paper/2411.01431/) (2024)
- [Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach](https://mlsanity.com/paper/2403.11380/) (2024)
- [Automated Dominative Subspace Mining for Efficient Neural Architecture Search](https://mlsanity.com/paper/2210.17180/) (2022)
- [Janus: An Algorithm-Evaluator Co-Evolution Framework for LLM-Driven Discovery under Expensive Evaluation Budgets](https://mlsanity.com/paper/2608.08189/) (2026)
- [ZeroLM: Data-Free Transformer Architecture Search for Language Models](https://mlsanity.com/paper/2503.18646/) (2025)
- [GreenMachine: Automatic Design of Zero-Cost Proxies for Energy-Efficient NAS](https://mlsanity.com/paper/2411.15290/) (2024)
- [Variation Matters: from Mitigating to Embracing Zero-Shot NAS Ranking Function Variation](https://mlsanity.com/paper/2502.19657/) (2025)
- [Bi-EZP: LLM-Guided Bilevel Program Evolution for Ensemble Zero-Cost Proxy Discovery](https://mlsanity.com/paper/2608.21927/) (2026)
- [ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search](https://mlsanity.com/paper/2609.14184/) (2026)
- [DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck](https://mlsanity.com/paper/2610.17184/) (2026)

## Abstract (arXiv)

Neural Architecture Search (NAS) has significantly improved productivity in the design and deployment of neural networks (NN). As NAS typically evaluates multiple models by training them partially or completely, the improved productivity comes at the cost of significant carbon footprint. To alleviate this expensive training routine, zero-shot/cost proxies analyze an NN at initialization to generate a score, which correlates highly with its true accuracy. Zero-cost proxies are currently designed by experts conducting multiple cycles of empirical testing on possible algorithms, datasets, and neural architecture design spaces. This experimentation lowers productivity and is an unsustainable approach towards zero-cost proxy design as deep learning use-cases diversify in nature. Additionally, existing zero-cost proxies fail to generalize across neural architecture design spaces. In this paper, we propose a genetic programming framework to automate the discovery of zero-cost proxies for neural architecture scoring. Our methodology efficiently discovers an interpretable and generalizable zero-cost proxy that gives state of the art score-accuracy correlation on all datasets and search spaces of NASBench-201 and Network Design Spaces (NDS). We believe that this research indicates a promising direction towards automatically discovering zero-cost proxies that can work across network architecture design spaces, datasets, and tasks.