# AutoHAS: Efficient Hyperparameter and Architecture Search

arXiv: https://arxiv.org/abs/2006.03656 · HTML: https://mlsanity.com/paper/2006.03656/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2020.

## Built on by (most cited first, 8 of 8)

- [PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning](https://mlsanity.com/paper/2306.12370/) (2023)
- [Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization](https://mlsanity.com/paper/2105.01015/) (2021)
- [Bi-level Alignment for Cross-Domain Crowd Counting](https://mlsanity.com/paper/2205.05844/) (2022)
- [DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture](https://mlsanity.com/paper/2109.05765/) (2021)
- [DAAS: Differentiable Architecture and Augmentation Policy Search](https://mlsanity.com/paper/2109.15273/) (2021)
- [Privacy-preserving Online AutoML for Domain-Specific Face Detection](https://mlsanity.com/paper/2203.08399/) (2022)
- [Meta-Learning of NAS for Few-shot Learning in Medical Image Applications](https://mlsanity.com/paper/2203.08951/) (2022)
- [Training Protocol Matters: Towards Accurate Scene Text Recognition via Training Protocol Searching](https://mlsanity.com/paper/2203.06696/) (2022)

## Abstract (arXiv)

Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architectures. In this work, we propose a unified pipeline, AutoHAS, to efficiently search for both architectures and hyperparameters. AutoHAS learns to alternately update the shared network weights and a reinforcement learning (RL) controller, which learns the probability distribution for the architecture candidates and HP candidates. A temporary weight is introduced to store the updated weight from the selected HPs (by the controller), and a validation accuracy based on this temporary weight serves as a reward to update the controller. In experiments, we show AutoHAS is efficient and generalizable to different search spaces, baselines and datasets. In particular, AutoHAS can improve the accuracy over popular network architectures, such as ResNet and EfficientNet, on CIFAR-10/100, ImageNet, and four more other datasets.