# Learning Multiple Defaults for Machine Learning Algorithms

arXiv: https://arxiv.org/abs/1811.09409 · HTML: https://mlsanity.com/paper/1811.09409/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2018.

## Built on by (most cited first, 10 of 10)

- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization](https://mlsanity.com/paper/2109.03670/) (2021)
- [Multi-Objective Model Selection for Time Series Forecasting](https://mlsanity.com/paper/2202.08485/) (2022)
- [Meta-Learning for Symbolic Hyperparameter Defaults](https://mlsanity.com/paper/2106.05767/) (2021)
- [The Role of Hyperparameters in Predictive Multiplicity](https://mlsanity.com/paper/2503.13506/) (2025)
- [Auto-nnU-Net: Towards Automated Medical Image Segmentation](https://mlsanity.com/paper/2505.16561/) (2025)
- [Consolidated learning -- a domain-specific model-free optimization strategy with examples for XGBoost and MIMIC-IV](https://mlsanity.com/paper/2201.11815/) (2022)
- [Finding One's Bearings in the Hyperparameter Landscape of a Wide-Kernel Convolutional Fault Detector](https://mlsanity.com/paper/2411.15191/) (2024)
- [Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Network](https://mlsanity.com/paper/2512.16491/) (2025)
- [Have I Solved This Before? Retrieving Similar Segmentation Problems for Evolutionary Learning](https://mlsanity.com/paper/2606.08155/) (2026)

## Abstract (arXiv)

The performance of modern machine learning methods highly depends on their hyperparameter configurations. One simple way of selecting a configuration is to use default settings, often proposed along with the publication and implementation of a new algorithm. Those default values are usually chosen in an ad-hoc manner to work good enough on a wide variety of datasets. To address this problem, different automatic hyperparameter configuration algorithms have been proposed, which select an optimal configuration per dataset. This principled approach usually improves performance but adds additional algorithmic complexity and computational costs to the training procedure. As an alternative to this, we propose learning a set of complementary default values from a large database of prior empirical results. Selecting an appropriate configuration on a new dataset then requires only a simple, efficient and embarrassingly parallel search over this set. We demonstrate the effectiveness and efficiency of the approach we propose in comparison to random search and Bayesian Optimization.