# Integrating Hyperparameter Search into Model-Free AutoML with Context-Free Grammars

arXiv: https://arxiv.org/abs/2404.03419 · HTML: https://mlsanity.com/paper/2404.03419/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 7 of 7)

- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [AMLB: an AutoML Benchmark](https://mlsanity.com/paper/2207.12560/) (2022)
- [AlphaD3M: Machine Learning Pipeline Synthesis](https://mlsanity.com/paper/2111.02508/) (2021)
- [Thompson sampling with the online bootstrap](https://mlsanity.com/paper/1410.4009/) (2014)
- [Automatic Machine Learning by Pipeline Synthesis using Model-Based Reinforcement Learning and a Grammar](https://mlsanity.com/paper/1905.10345/) (2019)
- [An Adaptive and Near Parameter-free Evolutionary Computation Approach Towards True Automation in AutoML](https://mlsanity.com/paper/2001.10178/) (2020)
- [A General Recipe for Automated Machine Learning in Practice](https://mlsanity.com/paper/2308.15647/) (2023)

## Abstract (arXiv)

Automated Machine Learning (AutoML) has become increasingly popular in recent years due to its ability to reduce the amount of time and expertise required to design and develop machine learning systems. This is very important for the practice of machine learning, as it allows building strong baselines quickly, improving the efficiency of the data scientists, and reducing the time to production. However, despite the advantages of AutoML, it faces several challenges, such as defining the solutions space and exploring it efficiently. Recently, some approaches have been shown to be able to do it using tree-based search algorithms and context-free grammars. In particular, GramML presents a model-free reinforcement learning approach that leverages pipeline configuration grammars and operates using Monte Carlo tree search. However, one of the limitations of GramML is that it uses default hyperparameters, limiting the search problem to finding optimal pipeline structures for the available data preprocessors and models. In this work, we propose an extension to GramML that supports larger search spaces including hyperparameter search. We evaluated the approach using an OpenML benchmark and found significant improvements compared to other state-of-the-art techniques.