# From Regression to Inference: Meta-Learning Predictors for Neural Architecture Search

arXiv: https://arxiv.org/abs/2605.09290 · HTML: https://mlsanity.com/paper/2605.09290/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 10 May 2026.

## Builds on (most cited first, 30 of 30)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [SMASH: One-Shot Model Architecture Search through HyperNetworks](https://mlsanity.com/paper/1708.05344/) (2017)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [Neural Predictor for Neural Architecture Search](https://mlsanity.com/paper/1912.00848/) (2019)
- [BRP-NAS: Prediction-based NAS using GCNs](https://mlsanity.com/paper/2007.08668/) (2020)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [Meta-Learning of Neural Architectures for Few-Shot Learning](https://mlsanity.com/paper/1911.11090/) (2019)
- [A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS](https://mlsanity.com/paper/2004.01899/) (2020)
- [Few-shot Neural Architecture Search](https://mlsanity.com/paper/2006.06863/) (2020)
- [A Study on Encodings for Neural Architecture Search](https://mlsanity.com/paper/2007.04965/) (2020)
- [A Semi-Supervised Assessor of Neural Architectures](https://mlsanity.com/paper/2005.06821/) (2020)
- [Peephole: Predicting Network Performance Before Training](https://mlsanity.com/paper/1712.03351/) (2017)
- [Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning](https://mlsanity.com/paper/2203.09137/) (2022)
- [Meta Architecture Search](https://mlsanity.com/paper/1812.09584/) (2018)
- [Homogeneous Architecture Augmentation for Neural Predictor](https://mlsanity.com/paper/2107.13153/) (2021)
- [Generic Neural Architecture Search via Regression](https://mlsanity.com/paper/2108.01899/) (2021)
- [TAPAS: Train-less Accuracy Predictor for Architecture Search](https://mlsanity.com/paper/1806.00250/) (2018)
- [Encodings for Prediction-based Neural Architecture Search](https://mlsanity.com/paper/2403.02484/) (2024)
- [Neural Architecture Performance Prediction Using Graph Neural Networks](https://mlsanity.com/paper/2010.10024/) (2020)
- [CAP: A Context-Aware Neural Predictor for NAS](https://mlsanity.com/paper/2406.02056/) (2024)
- [CARL: Causality-guided Architecture Representation Learning for an Interpretable Performance Predictor](https://mlsanity.com/paper/2506.04001/) (2025)

## Abstract (arXiv)

Prediction-based approaches are widely used in neural architecture search (NAS), where a predictor estimates the performance of candidate architectures to guide selection. However, existing predictors are typically trained via supervised regression on limited samples, leading to overfitting and poor generalization to unseen architectures. In this work, we propose a fundamentally different formulation that models performance prediction as a conditional function inference problem using a Convolutional Neural Process (ConvNP) with meta-learning capabilities. Instead of fitting a fixed mapping to limited samples, our approach meta-learns to infer performance from partial observations by training with context-target splits across a group of synthesized tasks, explicitly optimizing for generalization under data scarcity and aligning the training procedure with the deployment setting in NAS. We further design simple yet effective meta-features for cell-based architectures and evaluate our method on NAS-Bench-101 and NAS-Bench-201. Extensive experiments show that our approach consistently improves top-K ranking quality and achieves the state-of-the-art architecture selection using limited samples.