# Hyperparameter Importance for Machine Learning Algorithms

arXiv: https://arxiv.org/abs/2201.05132 · HTML: https://mlsanity.com/paper/2201.05132/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 20 references that are in the map.
- First posted on arXiv 13 Jan 2022.

## Builds on (most cited first, 3 of 3)

- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Tunability: Importance of Hyperparameters of Machine Learning Algorithms](https://mlsanity.com/paper/1802.09596/) (2018)
- [Hyperparameter Importance Across Datasets](https://mlsanity.com/paper/1710.04725/) (2017)

## Built on by (most cited first, 4 of 4)

- [Position: A Call to Action for a Human-Centered AutoML Paradigm](https://mlsanity.com/paper/2406.03348/) (2024)
- [MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters](https://mlsanity.com/paper/2402.02342/) (2024)
- [Behavior of Hyper-Parameters for Selected Machine Learning Algorithms: An Empirical Investigation](https://mlsanity.com/paper/2211.08536/) (2022)
- [Explaining AutoClustering: Uncovering Meta-Feature Contribution in AutoML for Clustering](https://mlsanity.com/paper/2602.18348/) (2026)

## Abstract (arXiv)

Hyperparameter plays an essential role in the fitting of supervised machine learning algorithms. However, it is computationally expensive to tune all the tunable hyperparameters simultaneously especially for large data sets. In this paper, we give a definition of hyperparameter importance that can be estimated by subsampling procedures. According to the importance, hyperparameters can then be tuned on the entire data set more efficiently. We show theoretically that the proposed importance on subsets of data is consistent with the one on the population data under weak conditions. Numerical experiments show that the proposed importance is consistent and can save a lot of computational resources.