mlsanityData to 7 Oct 2026

Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization

arXiv 2206.01409 ↗First posted 3 Jun 2022cs.LG math.ST stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture's integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 18 of its 66 references that are in the map. First posted on arXiv 3 Jun 2022.

Builds on

18 of 66 references.

References · 18

  1. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 65
  2. 2019Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variablescited by 64
  3. 2019Bayesian Optimisation over Multiple Continuous and Categorical Inputscited by 55
  4. 2018A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factorscited by 30
  5. 2018Similarity encoding for learning with dirty categorical variablescited by 28
  6. 2019Bayesian Optimization for Categorical and Category-Specific Continuous Inputscited by 28
  7. 2021Bayesian Optimization over Hybrid Spacescited by 23
  8. 2014Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spacescited by 16
  9. 2019Automated Machine Learning with Monte-Carlo Tree Searchcited by 8
  10. 2020Continuous surrogate-based optimization algorithms are well-suited for expensive discrete problemscited by 8
  11. 2021Non-smooth Bayesian Optimization in Tuning Problemscited by 8
  12. 2019Multitask and Transfer Learning for Autotuning Exascale Applicationscited by 7
6 more
  1. 2017Optimization by gradient boostingcited by 5
  2. 2018Progressive Sampling-Based Bayesian Optimization for Efficient and Automatic Machine Learning Model Selectioncited by 5
  3. 2019Black-box Combinatorial Optimization using Models with Integer-valued Minimacited by 5
  4. 2020Bandits with adversarial scalingcited by 5
  5. 2021Mixed Variable Bayesian Optimization with Frequency Modulated Kernelscited by 4
  6. 2022EzGP: Easy-to-Interpret Gaussian Process Models for Computer Experiments with Both Quantitative and Qualitative Factorscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Compactly-supported nonstationary kernels for computing exact Gaussian processes on big datacited by 2
  2. 2024Efficient Decision Trees for Tensor Regressionscited by 1
  3. 2026GGMPs: Generalized Gaussian Mixture Processescited by 1
  4. 2025Gradient-based Sample Selection for Faster Bayesian Optimizationcited by 0