# Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization

arXiv: https://arxiv.org/abs/2206.01409 · HTML: https://mlsanity.com/paper/2206.01409/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 66 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 18 of 18)

- [Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1706.03673/) (2017)
- [Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables](https://mlsanity.com/paper/1910.01688/) (2019)
- [Bayesian Optimisation over Multiple Continuous and Categorical Inputs](https://mlsanity.com/paper/1906.08878/) (2019)
- [A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors](https://mlsanity.com/paper/1806.07504/) (2018)
- [Similarity encoding for learning with dirty categorical variables](https://mlsanity.com/paper/1806.00979/) (2018)
- [Bayesian Optimization for Categorical and Category-Specific Continuous Inputs](https://mlsanity.com/paper/1911.12473/) (2019)
- [Bayesian Optimization over Hybrid Spaces](https://mlsanity.com/paper/2106.04682/) (2021)
- [Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces](https://mlsanity.com/paper/1409.4011/) (2014)
- [Automated Machine Learning with Monte-Carlo Tree Search](https://mlsanity.com/paper/1906.00170/) (2019)
- [Continuous surrogate-based optimization algorithms are well-suited for expensive discrete problems](https://mlsanity.com/paper/2011.03431/) (2020)
- [Non-smooth Bayesian Optimization in Tuning Problems](https://mlsanity.com/paper/2109.07563/) (2021)
- [Multitask and Transfer Learning for Autotuning Exascale Applications](https://mlsanity.com/paper/1908.05792/) (2019)
- [Optimization by gradient boosting](https://mlsanity.com/paper/1707.05023/) (2017)
- [Progressive Sampling-Based Bayesian Optimization for Efficient and Automatic Machine Learning Model Selection](https://mlsanity.com/paper/1812.02855/) (2018)
- [Black-box Combinatorial Optimization using Models with Integer-valued Minima](https://mlsanity.com/paper/1911.08817/) (2019)
- [Bandits with adversarial scaling](https://mlsanity.com/paper/2003.02287/) (2020)
- [Mixed Variable Bayesian Optimization with Frequency Modulated Kernels](https://mlsanity.com/paper/2102.12792/) (2021)
- [EzGP: Easy-to-Interpret Gaussian Process Models for Computer Experiments with Both Quantitative and Qualitative Factors](https://mlsanity.com/paper/2203.10130/) (2022)

## Built on by (most cited first, 4 of 4)

- [Compactly-supported nonstationary kernels for computing exact Gaussian processes on big data](https://mlsanity.com/paper/2411.05869/) (2024)
- [Efficient Decision Trees for Tensor Regressions](https://mlsanity.com/paper/2408.01926/) (2024)
- [GGMPs: Generalized Gaussian Mixture Processes](https://mlsanity.com/paper/2603.10442/) (2026)
- [Gradient-based Sample Selection for Faster Bayesian Optimization](https://mlsanity.com/paper/2504.07742/) (2025)

## Abstract (arXiv)

This paper presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture's integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization.