# Importance of Tuning Hyperparameters of Machine Learning Algorithms

arXiv: https://arxiv.org/abs/2007.07588 · HTML: https://mlsanity.com/paper/2007.07588/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2020.

## Built on by (most cited first, 20 of 20)

- [AMLB: an AutoML Benchmark](https://mlsanity.com/paper/2207.12560/) (2022)
- [Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML](https://mlsanity.com/paper/2303.08485/) (2023)
- [AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Models](https://mlsanity.com/paper/2408.00665/) (2024)
- [Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation](https://mlsanity.com/paper/2208.12866/) (2022)
- [Meta-Learning for Symbolic Hyperparameter Defaults](https://mlsanity.com/paper/2106.05767/) (2021)
- [Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices](https://mlsanity.com/paper/2506.04553/) (2025)
- [Exploring Opportunistic Meta-knowledge to Reduce Search Spaces for Automated Machine Learning](https://mlsanity.com/paper/2105.00282/) (2021)
- [High Per Parameter: A Large-Scale Study of Hyperparameter Tuning for Machine Learning Algorithms](https://mlsanity.com/paper/2207.06028/) (2022)
- [Enhancing supply chain security with automated machine learning](https://mlsanity.com/paper/2406.13166/) (2024)
- [Boosting Resource-Constrained Federated Learning Systems with Guessed Updates](https://mlsanity.com/paper/2110.11486/) (2021)
- [On Taking Advantage of Opportunistic Meta-knowledge to Reduce Configuration Spaces for Automated Machine Learning](https://mlsanity.com/paper/2208.04376/) (2022)
- [Behavior of Hyper-Parameters for Selected Machine Learning Algorithms: An Empirical Investigation](https://mlsanity.com/paper/2211.08536/) (2022)
- [FairPilot: An Explorative System for Hyperparameter Tuning through the Lens of Fairness](https://mlsanity.com/paper/2304.04679/) (2023)
- [Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes](https://mlsanity.com/paper/2306.07741/) (2023)
- [CloudSense: A Model for Cloud Type Identification using Machine Learning from Radar data](https://mlsanity.com/paper/2405.05988/) (2024)
- [Evaluating the Efficacy of Vectocardiographic and ECG Parameters for Efficient Tertiary Cardiology Care Allocation Using Decision Tree Analysis](https://mlsanity.com/paper/2412.11839/) (2024)
- [Data Driven Insights into Composition Property Relationships in FCC High Entropy Alloys](https://mlsanity.com/paper/2508.04841/) (2025)
- [Synthetic Data: AI's New Weapon Against Android Malware](https://mlsanity.com/paper/2511.19649/) (2025)
- [Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule](https://mlsanity.com/paper/2606.29119/) (2026)
- [On Hyperparameter Tuning on the Test Set](https://mlsanity.com/paper/2610.05902/) (2026)

## Abstract (arXiv)

The performance of many machine learning algorithms depends on their hyperparameter settings. The goal of this study is to determine whether it is important to tune a hyperparameter or whether it can be safely set to a default value. We present a methodology to determine the importance of tuning a hyperparameter based on a non-inferiority test and tuning risk: the performance loss that is incurred when a hyperparameter is not tuned, but set to a default value. Because our methods require the notion of a default parameter, we present a simple procedure that can be used to determine reasonable default parameters. We apply our methods in a benchmark study using 59 datasets from OpenML. Our results show that leaving particular hyperparameters at their default value is non-inferior to tuning these hyperparameters. In some cases, leaving the hyperparameter at its default value even outperforms tuning it using a search procedure with a limited number of iterations.