# Prior-Guided One-shot Neural Architecture Search

arXiv: https://arxiv.org/abs/2206.13329 · HTML: https://mlsanity.com/paper/2206.13329/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 19 of 19)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation](https://mlsanity.com/paper/1901.02985/) (2019)
- [Neural Architecture Search without Training](https://mlsanity.com/paper/2006.04647/) (2020)
- [BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models](https://mlsanity.com/paper/2003.11142/) (2020)
- [Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective](https://mlsanity.com/paper/2102.11535/) (2021)
- [FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search](https://mlsanity.com/paper/1907.01845/) (2019)
- [NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size](https://mlsanity.com/paper/2009.00437/) (2020)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [One-Shot Neural Architecture Search via Self-Evaluated Template Network](https://mlsanity.com/paper/1910.05733/) (2019)
- [BlockQNN: Efficient Block-wise Neural Network Architecture Generation](https://mlsanity.com/paper/1808.05584/) (2018)
- [Learning Efficient, Explainable and Discriminative Representations for Pulmonary Nodules Classification](https://mlsanity.com/paper/2101.07429/) (2021)
- [Balanced One-shot Neural Architecture Optimization](https://mlsanity.com/paper/1909.10815/) (2019)
- [Improving Ranking Correlation of Supernet with Candidates Enhancement and Progressive Training](https://mlsanity.com/paper/2108.05866/) (2021)

## Built on by (most cited first, 12 of 12)

- [Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models](https://mlsanity.com/paper/2406.02924/) (2024)
- [DisWOT: Student Architecture Search for Distillation WithOut Training](https://mlsanity.com/paper/2303.15678/) (2023)
- [DEHRFormer: Real-time Transformer for Depth Estimation and Haze Removal from Varicolored Haze Scenes](https://mlsanity.com/paper/2303.06905/) (2023)
- [GP-NAS-ensemble: a model for NAS Performance Prediction](https://mlsanity.com/paper/2301.09231/) (2023)
- [RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxies](https://mlsanity.com/paper/2301.09850/) (2023)
- [Pruning Large Language Models via Accuracy Predictor](https://mlsanity.com/paper/2309.09507/) (2023)
- [Efficiently Distilling LLMs for Edge Applications](https://mlsanity.com/paper/2404.01353/) (2024)
- [Transfer-Once-For-All: AI Model Optimization for Edge](https://mlsanity.com/paper/2303.15485/) (2023)
- [InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning](https://mlsanity.com/paper/2308.15609/) (2023)
- [TVT: Training-Free Vision Transformer Search on Tiny Datasets](https://mlsanity.com/paper/2311.14337/) (2023)
- [A Survey on Recent Teacher-student Learning Studies](https://mlsanity.com/paper/2304.04615/) (2023)
- [W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models](https://mlsanity.com/paper/2504.15983/) (2025)

## Abstract (arXiv)

Neural architecture search methods seek optimal candidates with efficient weight-sharing supernet training. However, recent studies indicate poor ranking consistency about the performance between stand-alone architectures and shared-weight networks. In this paper, we present Prior-Guided One-shot NAS (PGONAS) to strengthen the ranking correlation of supernets. Specifically, we first explore the effect of activation functions and propose a balanced sampling strategy based on the Sandwich Rule to alleviate weight coupling in the supernet. Then, FLOPs and Zen-Score are adopted to guide the training of supernet with ranking correlation loss. Our PGONAS ranks 3rd place in the supernet Track Track of CVPR2022 Second lightweight NAS challenge. Code is available in https://github.com/pprp/CVPR2022-NAS?competition-Track1-3th-solution.