# Bayesian Optimisation over Multiple Continuous and Categorical Inputs

arXiv: https://arxiv.org/abs/1906.08878 · HTML: https://mlsanity.com/paper/1906.08878/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 30 of 56)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces](https://mlsanity.com/paper/2102.07188/) (2021)
- [Revisiting Design Choices in Offline Model-Based Reinforcement Learning](https://mlsanity.com/paper/2110.04135/) (2021)
- [GIBBON: General-purpose Information-Based Bayesian OptimisatioN](https://mlsanity.com/paper/2102.03324/) (2021)
- [Joint Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/2210.02905/) (2022)
- [Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization](https://mlsanity.com/paper/2210.10199/) (2022)
- [Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spaces](https://mlsanity.com/paper/2307.00618/) (2023)
- [Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings](https://mlsanity.com/paper/2303.01774/) (2023)
- [Bayesian Optimization over Hybrid Spaces](https://mlsanity.com/paper/2106.04682/) (2021)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Counterfactual Explanations for Arbitrary Regression Models](https://mlsanity.com/paper/2106.15212/) (2021)
- [Distributionally Robust Bayesian Optimization with varphi-divergences](https://mlsanity.com/paper/2203.02128/) (2022)
- [Framework and Benchmarks for Combinatorial and Mixed-variable Bayesian Optimization](https://mlsanity.com/paper/2306.09803/) (2023)
- [Bayesian optimization as a flexible and efficient design framework for sustainable process systems](https://mlsanity.com/paper/2401.16373/) (2024)
- [Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces](https://mlsanity.com/paper/2207.00879/) (2022)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow](https://mlsanity.com/paper/2302.08436/) (2023)
- [EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functions](https://mlsanity.com/paper/2106.04618/) (2021)
- [c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization](https://mlsanity.com/paper/2211.14411/) (2022)
- [Non-smooth Bayesian Optimization in Tuning Problems](https://mlsanity.com/paper/2109.07563/) (2021)
- [SOBER: Highly Parallel Bayesian Optimization and Bayesian Quadrature over Discrete and Mixed Spaces](https://mlsanity.com/paper/2301.11832/) (2023)
- [DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture](https://mlsanity.com/paper/2109.05765/) (2021)
- [Bayesian Optimization For Multi-Objective Mixed-Variable Problems](https://mlsanity.com/paper/2201.12767/) (2022)
- [AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuning](https://mlsanity.com/paper/2602.22268/) (2026)
- [Mixed Variable Bayesian Optimization with Frequency Modulated Kernels](https://mlsanity.com/paper/2102.12792/) (2021)
- [Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization](https://mlsanity.com/paper/2206.01409/) (2022)
- [CAGES: Cost-Aware Gradient Entropy Search for Efficient Local Multi-Fidelity Bayesian Optimization](https://mlsanity.com/paper/2405.07760/) (2024)
- [B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search](https://mlsanity.com/paper/2202.03005/) (2022)
- [Mastering the exploration-exploitation trade-off in Bayesian Optimization](https://mlsanity.com/paper/2305.08624/) (2023)
- [A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Lifting](https://mlsanity.com/paper/2404.12219/) (2024)

## Abstract (arXiv)

Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select values for both categorical and continuous inputs. We model this mixed-type space using a Gaussian Process kernel, designed to allow sharing of information across multiple categorical variables, each with multiple possible values; this allows CoCaBO to leverage all available data efficiently. We extend our method to the batch setting and propose an efficient selection procedure that dynamically balances exploration and exploitation whilst encouraging batch diversity. We demonstrate empirically that our method outperforms existing approaches on both synthetic and real-world optimisation tasks with continuous and categorical inputs.