# Hyperparameter Importance Across Datasets

arXiv: https://arxiv.org/abs/1710.04725 · HTML: https://mlsanity.com/paper/1710.04725/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2017.

## Built on by (most cited first, 30 of 63)

- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [Automated Reinforcement Learning (AutoRL): A Survey and Open Problems](https://mlsanity.com/paper/2201.03916/) (2022)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [AMLB: an AutoML Benchmark](https://mlsanity.com/paper/2207.12560/) (2022)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Contextualize Me -- The Case for Context in Reinforcement Learning](https://mlsanity.com/paper/2202.04500/) (2022)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [Artificial intelligence to advance Earth observation: : A review of models, recent trends, and pathways forward](https://mlsanity.com/paper/2305.08413/) (2023)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning](https://mlsanity.com/paper/2110.02102/) (2021)
- [Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML](https://mlsanity.com/paper/2303.08485/) (2023)
- [Position: A Call to Action for a Human-Centered AutoML Paradigm](https://mlsanity.com/paper/2406.03348/) (2024)
- [Hyperparameter Importance Analysis for Multi-Objective AutoML](https://mlsanity.com/paper/2405.07640/) (2024)
- [Hyperparameter Importance of Quantum Neural Networks Across Small Datasets](https://mlsanity.com/paper/2206.09992/) (2022)
- [Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization](https://mlsanity.com/paper/2405.15393/) (2024)
- [Multi-Objective Optimization of Performance and Interpretability of Tabular Supervised Machine Learning Models](https://mlsanity.com/paper/2307.08175/) (2023)
- [Show Your Work with Confidence: Confidence Bands for Tuning Curves](https://mlsanity.com/paper/2311.09480/) (2023)
- [AutoXPCR: Automated Multi-Objective Model Selection for Time Series Forecasting](https://mlsanity.com/paper/2312.13038/) (2023)
- [Multi-Fidelity Methods for Optimization: A Survey](https://mlsanity.com/paper/2402.09638/) (2024)
- [Using AntiPatterns to avoid MLOps Mistakes](https://mlsanity.com/paper/2107.00079/) (2021)
- [Overtuning in Hyperparameter Optimization](https://mlsanity.com/paper/2506.19540/) (2025)
- [Meta-Learning for Symbolic Hyperparameter Defaults](https://mlsanity.com/paper/2106.05767/) (2021)
- [Hyperparameter Importance for Machine Learning Algorithms](https://mlsanity.com/paper/2201.05132/) (2022)
- [CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DAC](https://mlsanity.com/paper/2407.05789/) (2024)
- [HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization](https://mlsanity.com/paper/2502.01276/) (2025)
- [Beyond transparency: computational reliabilism as an externalist epistemology of algorithms](https://mlsanity.com/paper/2502.20402/) (2025)
- [The Role of Hyperparameters in Predictive Multiplicity](https://mlsanity.com/paper/2503.13506/) (2025)
- [Explaining the Performance of Multi-label Classification Methods with Data Set Properties](https://mlsanity.com/paper/2106.15411/) (2021)

## Abstract (arXiv)

With the advent of automated machine learning, automated hyperparameter optimization methods are by now routinely used in data mining. However, this progress is not yet matched by equal progress on automatic analyses that yield information beyond performance-optimizing hyperparameter settings. In this work, we aim to answer the following two questions: Given an algorithm, what are generally its most important hyperparameters, and what are typically good values for these? We present methodology and a framework to answer these questions based on meta-learning across many datasets. We apply this methodology using the experimental meta-data available on OpenML to determine the most important hyperparameters of support vector machines, random forests and Adaboost, and to infer priors for all their hyperparameters. The results, obtained fully automatically, provide a quantitative basis to focus efforts in both manual algorithm design and in automated hyperparameter optimization. The conducted experiments confirm that the hyperparameters selected by the proposed method are indeed the most important ones and that the obtained priors also lead to statistically significant improvements in hyperparameter optimization.