# EcoNAS: Finding Proxies for Economical Neural Architecture Search

arXiv: https://arxiv.org/abs/2001.01233 · HTML: https://mlsanity.com/paper/2001.01233/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jan 2020.

## Built on by (most cited first, 30 of 61)

- [Zero-Cost Proxies for Lightweight NAS](https://mlsanity.com/paper/2101.08134/) (2021)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [How Powerful are Performance Predictors in Neural Architecture Search?](https://mlsanity.com/paper/2104.01177/) (2021)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients](https://mlsanity.com/paper/2301.11300/) (2023)
- [GLiT: Neural Architecture Search for Global and Local Image Transformer](https://mlsanity.com/paper/2107.02960/) (2021)
- [Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets](https://mlsanity.com/paper/2107.00860/) (2021)
- [KNAS: Green Neural Architecture Search](https://mlsanity.com/paper/2111.13293/) (2021)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search](https://mlsanity.com/paper/2105.10154/) (2021)
- [ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild](https://mlsanity.com/paper/2208.11547/) (2022)
- [ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies](https://mlsanity.com/paper/2110.10423/) (2021)
- [BN-NAS: Neural Architecture Search with Batch Normalization](https://mlsanity.com/paper/2108.07375/) (2021)
- [Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2307.01998/) (2023)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [GradSign: Model Performance Inference with Theoretical Insights](https://mlsanity.com/paper/2110.08616/) (2021)
- [Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?](https://mlsanity.com/paper/2203.14577/) (2022)
- [EZNAS: Evolving Zero Cost Proxies For Neural Architecture Scoring](https://mlsanity.com/paper/2209.07413/) (2022)
- [Accelerating Neural Architecture Search via Proxy Data](https://mlsanity.com/paper/2106.04784/) (2021)
- [Generic Neural Architecture Search via Regression](https://mlsanity.com/paper/2108.01899/) (2021)
- [Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localization](https://mlsanity.com/paper/2201.08613/) (2022)
- [PASHA: Efficient HPO and NAS with Progressive Resource Allocation](https://mlsanity.com/paper/2207.06940/) (2022)
- [Neural Architecture Search for Dense Prediction Tasks in Computer Vision](https://mlsanity.com/paper/2202.07242/) (2022)
- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS](https://mlsanity.com/paper/2403.04161/) (2024)
- [Zero-Cost Operation Scoring in Differentiable Architecture Search](https://mlsanity.com/paper/2106.06799/) (2021)
- [Training-free Neural Architecture Search for RNNs and Transformers](https://mlsanity.com/paper/2306.00288/) (2023)
- [FL-NAS: Towards Fairness of NAS for Resource Constrained Devices via Large Language Models](https://mlsanity.com/paper/2402.06696/) (2024)
- [Encodings for Prediction-based Neural Architecture Search](https://mlsanity.com/paper/2403.02484/) (2024)
- [Task-Adaptive Neural Network Search with Meta-Contrastive Learning](https://mlsanity.com/paper/2103.01495/) (2021)

## Abstract (arXiv)

Neural Architecture Search (NAS) achieves significant progress in many computer vision tasks. While many methods have been proposed to improve the efficiency of NAS, the search progress is still laborious because training and evaluating plausible architectures over large search space is time-consuming. Assessing network candidates under a proxy (i.e., computationally reduced setting) thus becomes inevitable. In this paper, we observe that most existing proxies exhibit different behaviors in maintaining the rank consistency among network candidates. In particular, some proxies can be more reliable -- the rank of candidates does not differ much comparing their reduced setting performance and final performance. In this paper, we systematically investigate some widely adopted reduction factors and report our observations. Inspired by these observations, we present a reliable proxy and further formulate a hierarchical proxy strategy. The strategy spends more computations on candidate networks that are potentially more accurate, while discards unpromising ones in early stage with a fast proxy. This leads to an economical evolutionary-based NAS (EcoNAS), which achieves an impressive 400x search time reduction in comparison to the evolutionary-based state of the art (8 vs. 3150 GPU days). Some new proxies led by our observations can also be applied to accelerate other NAS methods while still able to discover good candidate networks with performance matching those found by previous proxy strategies.