# HyperSTAR: Task-Aware Hyperparameters for Deep Networks

arXiv: https://arxiv.org/abs/2005.10524 · HTML: https://mlsanity.com/paper/2005.10524/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2020.

## Built on by (most cited first, 9 of 9)

- [MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks](https://mlsanity.com/paper/2304.14979/) (2023)
- [T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging](https://mlsanity.com/paper/2111.07535/) (2021)
- [Making Reconstruction-based Method Great Again for Video Anomaly Detection](https://mlsanity.com/paper/2301.12048/) (2023)
- [Context-Aware Streaming Perception in Dynamic Environments](https://mlsanity.com/paper/2208.07479/) (2022)
- [NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models](https://mlsanity.com/paper/2108.03434/) (2021)
- [DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture](https://mlsanity.com/paper/2109.05765/) (2021)
- [Amortized Auto-Tuning: Cost-Efficient Bayesian Transfer Optimization for Hyperparameter Recommendation](https://mlsanity.com/paper/2106.09179/) (2021)
- [Privacy-preserving Online AutoML for Domain-Specific Face Detection](https://mlsanity.com/paper/2203.08399/) (2022)
- [Filter Distribution Templates in Convolutional Networks for Image Classification Tasks](https://mlsanity.com/paper/2104.13993/) (2021)

## Abstract (arXiv)

While deep neural networks excel in solving visual recognition tasks, they require significant effort to find hyperparameters that make them work optimally. Hyperparameter Optimization (HPO) approaches have automated the process of finding good hyperparameters but they do not adapt to a given task (task-agnostic), making them computationally inefficient. To reduce HPO time, we present HyperSTAR (System for Task Aware Hyperparameter Recommendation), a task-aware method to warm-start HPO for deep neural networks. HyperSTAR ranks and recommends hyperparameters by predicting their performance conditioned on a joint dataset-hyperparameter space. It learns a dataset (task) representation along with the performance predictor directly from raw images in an end-to-end fashion. The recommendations, when integrated with an existing HPO method, make it task-aware and significantly reduce the time to achieve optimal performance. We conduct extensive experiments on 10 publicly available large-scale image classification datasets over two different network architectures, validating that HyperSTAR evaluates 50% less configurations to achieve the best performance compared to existing methods. We further demonstrate that HyperSTAR makes Hyperband (HB) task-aware, achieving the optimal accuracy in just 25% of the budget required by both vanilla HB and Bayesian Optimized HB~(BOHB).