# A Meta-Level Learning Algorithm for Sequential Hyper-Parameter Space Reduction in AutoML

arXiv: https://arxiv.org/abs/2312.06305 · HTML: https://mlsanity.com/paper/2312.06305/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 32 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 5 of 5)

- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Meta-Learning: A Survey](https://mlsanity.com/paper/1810.03548/) (2018)
- [Bootstrapping the Out-of-sample Predictions for Efficient and Accurate Cross-Validation](https://mlsanity.com/paper/1708.07180/) (2017)
- [Feature Selection with the R Package MXM: Discovering Statistically-Equivalent Feature Subsets](https://mlsanity.com/paper/1611.03227/) (2016)
- [AutonoML: Towards an Integrated Framework for Autonomous Machine Learning](https://mlsanity.com/paper/2012.12600/) (2020)

## Built on by (most cited first, 1 of 1)

- [LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection](https://mlsanity.com/paper/2510.26510/) (2025)

## Abstract (arXiv)

AutoML platforms have numerous options for the algorithms to try for each step of the analysis, i.e., different possible algorithms for imputation, transformations, feature selection, and modelling. Finding the optimal combination of algorithms and hyper-parameter values is computationally expensive, as the number of combinations to explore leads to an exponential explosion of the space. In this paper, we present the Sequential Hyper-parameter Space Reduction (SHSR) algorithm that reduces the space for an AutoML tool with negligible drop in its predictive performance. SHSR is a meta-level learning algorithm that analyzes past runs of an AutoML tool on several datasets and learns which hyper-parameter values to filter out from consideration on a new dataset to analyze. SHSR is evaluated on 284 classification and 375 regression problems, showing an approximate 30% reduction in execution time with a performance drop of less than 0.1%.