# A comparison of mixed-variables Bayesian optimization approaches

arXiv: https://arxiv.org/abs/2111.01533 · HTML: https://mlsanity.com/paper/2111.01533/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 38 references that are in the map.
- First posted on arXiv 30 Oct 2021.

## Builds on (most cited first, 10 of 10)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [Maximizing acquisition functions for Bayesian optimization](https://mlsanity.com/paper/1805.10196/) (2018)
- [Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables](https://mlsanity.com/paper/1910.01688/) (2019)
- [A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors](https://mlsanity.com/paper/1806.07504/) (2018)
- [Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian](https://mlsanity.com/paper/1605.09466/) (2016)
- [Group kernels for Gaussian process metamodels with categorical inputs](https://mlsanity.com/paper/1802.02368/) (2018)
- [mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions](https://mlsanity.com/paper/1703.03373/) (2017)
- [Efficient global optimization of constrained mixed variable problems](https://mlsanity.com/paper/1806.03975/) (2018)
- [Revisiting Bayesian Optimization in the light of the COCO benchmark](https://mlsanity.com/paper/2103.16649/) (2021)

## Built on by (most cited first, 6 of 6)

- [SMT 2.0: A Surrogate Modeling Toolbox with a focus on Hierarchical and Mixed Variables Gaussian Processes](https://mlsanity.com/paper/2305.13998/) (2023)
- [A mixed-categorical correlation kernel for Gaussian process](https://mlsanity.com/paper/2211.08262/) (2022)
- [BaCO: A Fast and Portable Bayesian Compiler Optimization Framework](https://mlsanity.com/paper/2212.11142/) (2022)
- [High-dimensional mixed-categorical Gaussian processes with application to multidisciplinary design optimization for a green aircraft](https://mlsanity.com/paper/2311.06130/) (2023)
- [Mastering the exploration-exploitation trade-off in Bayesian Optimization](https://mlsanity.com/paper/2305.08624/) (2023)
- [CatBOX: A Categorical-Continuous Bayesian Optimization with Spectral Mixture Kernels for Accelerated Catalysis Experiments](https://mlsanity.com/paper/2505.17393/) (2025)

## Abstract (arXiv)

Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation.General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientific question. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. The continuous space is more easily harvested by classical Bayesian optimization techniques than a mixed space would. Discrete variables are recovered either subsequently to the continuous optimization, or simultaneously with an additional continuous-discrete compatibility constraint that is handled with augmented Lagrangians. Several possible implementations of such Bayesian mixed optimizers are compared. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space. Among the algorithms involving latent variables and an augmented Lagrangian, a particular attention is devoted to the Lagrange multipliers for which a local and a global estimation techniques are studied. The comparisons are based on the repeated optimization of three analytical functions and a beam design problem.