# A resource-efficient method for repeated HPO and NAS problems

arXiv: https://arxiv.org/abs/2103.16111 · HTML: https://mlsanity.com/paper/2103.16111/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [Non-stochastic Best Arm Identification and Hyperparameter Optimization](https://mlsanity.com/paper/1502.07943/) (2015)
- [Task-Free Continual Learning](https://mlsanity.com/paper/1812.03596/) (2018)
- [Coresets via Bilevel Optimization for Continual Learning and Streaming](https://mlsanity.com/paper/2006.03875/) (2020)
- [Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization](https://mlsanity.com/paper/1905.04970/) (2019)
- [Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning](https://mlsanity.com/paper/1909.12552/) (2019)
- [Hyperparameter Transfer Across Developer Adjustments](https://mlsanity.com/paper/2010.13117/) (2020)
- [Practical and sample efficient zero-shot HPO](https://mlsanity.com/paper/2007.13382/) (2020)
- [A Quantile-based Approach for Hyperparameter Transfer Learning](https://mlsanity.com/paper/1909.13595/) (2019)

## Built on by (most cited first, 5 of 5)

- [PASHA: Efficient HPO and NAS with Progressive Resource Allocation](https://mlsanity.com/paper/2207.06940/) (2022)
- [Best Arm Identification with Resource Constraints](https://mlsanity.com/paper/2402.19090/) (2024)
- [Renate: A Library for Real-World Continual Learning](https://mlsanity.com/paper/2304.12067/) (2023)
- [Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimisation](https://mlsanity.com/paper/2306.16916/) (2023)
- [Learning with a Budget: Identifying the Best Arm with Resource Constraints](https://mlsanity.com/paper/2602.24146/) (2026)

## Abstract (arXiv)

In this work we consider the problem of repeated hyperparameter and neural architecture search (HNAS). We propose an extension of Successive Halving that is able to leverage information gained in previous HNAS problems with the goal of saving computational resources. We empirically demonstrate that our solution is able to drastically decrease costs while maintaining accuracy and being robust to negative transfer. Our method is significantly simpler than competing transfer learning approaches, setting a new baseline for transfer learning in HNAS.