# Model-based Asynchronous Hyperparameter and Neural Architecture Search

arXiv: https://arxiv.org/abs/2003.10865 · HTML: https://mlsanity.com/paper/2003.10865/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2020.

## Built on by (most cited first, 4 of 4)

- [Benchmarking Multimodal AutoML for Tabular Data with Text Fields](https://mlsanity.com/paper/2111.02705/) (2021)
- [Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale](https://mlsanity.com/paper/2201.06834/) (2022)
- [Structural Pruning of Pre-trained Language Models via Neural Architecture Search](https://mlsanity.com/paper/2405.02267/) (2024)
- [Automated Benchmark-Driven Design and Explanation of Hyperparameter Optimizers](https://mlsanity.com/paper/2111.14756/) (2021)

## Abstract (arXiv)

We introduce a model-based asynchronous multi-fidelity method for hyperparameter and neural architecture search that combines the strengths of asynchronous Hyperband and Gaussian process-based Bayesian optimization. At the heart of our method is a probabilistic model that can simultaneously reason across hyperparameters and resource levels, and supports decision-making in the presence of pending evaluations. We demonstrate the effectiveness of our method on a wide range of challenging benchmarks, for tabular data, image classification and language modelling, and report substantial speed-ups over current state-of-the-art methods. Our new methods, along with asynchronous baselines, are implemented in a distributed framework which will be open sourced along with this publication.