# Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Search

arXiv: https://arxiv.org/abs/2306.02459 · HTML: https://mlsanity.com/paper/2306.02459/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 4 Jun 2023.

## Builds on (most cited first, 24 of 24)

- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Pruning neural networks without any data by iteratively conserving synaptic flow](https://mlsanity.com/paper/2006.05467/) (2020)
- [Picking Winning Tickets Before Training by Preserving Gradient Flow](https://mlsanity.com/paper/2002.07376/) (2020)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [Neural Architecture Search without Training](https://mlsanity.com/paper/2006.04647/) (2020)
- [Zero-Cost Proxies for Lightweight NAS](https://mlsanity.com/paper/2101.08134/) (2021)
- [Neural Predictor for Neural Architecture Search](https://mlsanity.com/paper/1912.00848/) (2019)
- [BRP-NAS: Prediction-based NAS using GCNs](https://mlsanity.com/paper/2007.08668/) (2020)
- [HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark](https://mlsanity.com/paper/2103.10584/) (2021)
- [ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation](https://mlsanity.com/paper/1812.08934/) (2018)
- [NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search](https://mlsanity.com/paper/2007.10396/) (2020)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks](https://mlsanity.com/paper/2008.09777/) (2020)
- [TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search](https://mlsanity.com/paper/2105.11871/) (2021)
- [Evaluating Efficient Performance Estimators of Neural Architectures](https://mlsanity.com/paper/2008.03064/) (2020)
- [A Study on Encodings for Neural Architecture Search](https://mlsanity.com/paper/2007.04965/) (2020)
- [NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies](https://mlsanity.com/paper/2210.03230/) (2022)
- [BlockSwap: Fisher-guided Block Substitution for Network Compression on a Budget](https://mlsanity.com/paper/1906.04113/) (2019)
- [HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learning](https://mlsanity.com/paper/2106.08630/) (2021)
- [EZNAS: Evolving Zero Cost Proxies For Neural Architecture Scoring](https://mlsanity.com/paper/2209.07413/) (2022)
- [NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search](https://mlsanity.com/paper/2003.12857/) (2020)
- [MAPLE-Edge: A Runtime Latency Predictor for Edge Devices](https://mlsanity.com/paper/2204.12950/) (2022)

## Built on by (most cited first, 6 of 6)

- [Surprisingly Strong Performance Prediction with Neural Graph Features](https://mlsanity.com/paper/2404.16551/) (2024)
- [Regression Language Models for Code](https://mlsanity.com/paper/2509.26476/) (2025)
- [Encodings for Prediction-based Neural Architecture Search](https://mlsanity.com/paper/2403.02484/) (2024)
- [On Latency Predictors for Neural Architecture Search](https://mlsanity.com/paper/2403.02446/) (2024)
- [HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS](https://mlsanity.com/paper/2607.11746/) (2026)
- [Rank Portability Does Not Imply Feasibility Portability: Target-Specific Evaluation of Joint Hardware Constraints](https://mlsanity.com/paper/2609.22122/) (2026)

## Abstract (arXiv)

Many hardware-aware neural architecture search (NAS) methods have been developed to optimize the topology of neural networks (NN) with the joint objectives of higher accuracy and lower latency. Recently, both accuracy and latency predictors have been used in NAS with great success, achieving high sample efficiency and accurate modeling of hardware (HW) device latency respectively. However, a new accuracy predictor needs to be trained for every new NAS search space or NN task, and a new latency predictor needs to be additionally trained for every new HW device. In this paper, we explore methods to enable multi-task, multi-search-space, and multi-HW adaptation of accuracy and latency predictors to reduce the cost of NAS. We introduce a novel search-space independent NN encoding based on zero-cost proxies that achieves sample-efficient prediction on multiple tasks and NAS search spaces, improving the end-to-end sample efficiency of latency and accuracy predictors by over an order of magnitude in multiple scenarios. For example, our NN encoding enables multi-search-space transfer of latency predictors from NASBench-201 to FBNet (and vice-versa) in under 85 HW measurements, a 400× improvement in sample efficiency compared to a recent meta-learning approach. Our method also improves the total sample efficiency of accuracy predictors by over an order of magnitude. Finally, we demonstrate the effectiveness of our method for multi-search-space and multi-task accuracy prediction on 28 NAS search spaces and tasks.