# Mixed-Variable Bayesian Optimization

arXiv: https://arxiv.org/abs/1907.01329 · HTML: https://mlsanity.com/paper/1907.01329/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2019.

## Built on by (most cited first, 29 of 29)

- [HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO](https://mlsanity.com/paper/2109.06716/) (2021)
- [Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces](https://mlsanity.com/paper/2102.07188/) (2021)
- [Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview](https://mlsanity.com/paper/2206.07438/) (2022)
- [Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization](https://mlsanity.com/paper/2210.10199/) (2022)
- [Sample-Then-Optimize Batch Neural Thompson Sampling](https://mlsanity.com/paper/2210.06850/) (2022)
- [Bayesian Optimization over Hybrid Spaces](https://mlsanity.com/paper/2106.04682/) (2021)
- [Counterfactual Explanations for Arbitrary Regression Models](https://mlsanity.com/paper/2106.15212/) (2021)
- [Bayesian optimization as a flexible and efficient design framework for sustainable process systems](https://mlsanity.com/paper/2401.16373/) (2024)
- [Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces](https://mlsanity.com/paper/2207.00879/) (2022)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization](https://mlsanity.com/paper/2211.14411/) (2022)
- [Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximation](https://mlsanity.com/paper/2410.16893/) (2024)
- [Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks](https://mlsanity.com/paper/2106.09884/) (2021)
- [Lossy compression of matrices by black-box optimisation of mixed integer nonlinear programming](https://mlsanity.com/paper/2204.10579/) (2022)
- [Bayesian Optimization For Multi-Objective Mixed-Variable Problems](https://mlsanity.com/paper/2201.12767/) (2022)
- [Mixed Variable Bayesian Optimization with Frequency Modulated Kernels](https://mlsanity.com/paper/2102.12792/) (2021)
- [Hidden-Variables Genetic Algorithm for Variable-Size Design Space Optimal Layout Problems with Application to Aerospace Vehicles](https://mlsanity.com/paper/2212.11011/) (2022)
- [BARK: A Fully Bayesian Tree Kernel for Black-box Optimization](https://mlsanity.com/paper/2503.05574/) (2025)
- [Fast Design Space Exploration of Nonlinear Systems: Part I](https://mlsanity.com/paper/2104.01747/) (2021)
- [Machine Learning for Combinatorial Optimisation of Partially-Specified Problems: Regret Minimisation as a Unifying Lens](https://mlsanity.com/paper/2205.10157/) (2022)
- [A Trajectory-Based Bayesian Approach to Multi-Objective Hyperparameter Optimization with Epoch-Aware Trade-Offs](https://mlsanity.com/paper/2405.15303/) (2024)
- [Function-on-Function Bayesian Optimization](https://mlsanity.com/paper/2511.12783/) (2025)
- [Automated Discovery of Laser Dicing Processes with Bayesian Optimization for Semiconductor Manufacturing](https://mlsanity.com/paper/2511.23141/) (2025)
- [Simulating, Fast and Slow: Learning Policies for Black-Box Optimization](https://mlsanity.com/paper/2406.04261/) (2024)
- [Bayesian Optimization for General Reaction Conditions](https://mlsanity.com/paper/2502.18966/) (2025)
- [Weighted Euclidean Distance Matrices over Mixed Continuous and Categorical Inputs for Gaussian Process Models](https://mlsanity.com/paper/2503.02630/) (2025)
- [Adaptive Linear Embedding for Nonstationary High-Dimensional Optimization](https://mlsanity.com/paper/2505.11281/) (2025)
- [Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents](https://mlsanity.com/paper/2507.13491/) (2025)
- [Beyond Heuristics: Globally Optimal Configuration of Implicit Neural Representations](https://mlsanity.com/paper/2509.23139/) (2025)

## Abstract (arXiv)

The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete domains have been widely studied. However, few methods exist for mixed-variable domains and none of them can handle discrete constraints that arise in many real-world applications. In this paper, we introduce MiVaBo, a novel BO algorithm for the efficient optimization of mixed-variable functions combining a linear surrogate model based on expressive feature representations with Thompson sampling. We propose an effective method to optimize its acquisition function, a challenging problem for mixed-variable domains, making MiVaBo the first BO method that can handle complex constraints over the discrete variables. Moreover, we provide the first convergence analysis of a mixed-variable BO algorithm. Finally, we show that MiVaBo is significantly more sample efficient than state-of-the-art mixed-variable BO algorithms on several hyperparameter tuning tasks, including the tuning of deep generative models.