# Bayesian Variational Optimization for Combinatorial Spaces

arXiv: https://arxiv.org/abs/2011.02004 · HTML: https://mlsanity.com/paper/2011.02004/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2020.

## Built on by (most cited first, 4 of 4)

- [DOCKSTRING: easy molecular docking yields better benchmarks for ligand design](https://mlsanity.com/paper/2110.15486/) (2021)
- [Lossy compression of matrices by black-box optimisation of mixed integer nonlinear programming](https://mlsanity.com/paper/2204.10579/) (2022)
- [Atom-by-atom protein generation and beyond with language models](https://mlsanity.com/paper/2308.09482/) (2023)
- [Tree-Structured Parzen Estimator Can Solve Black-Box Combinatorial Optimization More Efficiently](https://mlsanity.com/paper/2507.08053/) (2025)

## Abstract (arXiv)

This paper focuses on Bayesian Optimization in combinatorial spaces. In many applications in the natural science. Broad applications include the study of molecules, proteins, DNA, device structures and quantum circuit designs, a on optimization over combinatorial categorical spaces is needed to find optimal or pareto-optimal solutions. However, only a limited amount of methods have been proposed to tackle this problem. Many of them depend on employing Gaussian Process for combinatorial Bayesian Optimizations. Gaussian Processes suffer from scalability issues for large data sizes as their scaling is cubic with respect to the number of data points. This is often impractical for optimizing large search spaces. Here, we introduce a variational Bayesian optimization method that combines variational optimization and continuous relaxations to the optimization of the acquisition function for Bayesian optimization. Critically, this method allows for gradient-based optimization and has the capability of optimizing problems with large data size and data dimensions. We have shown the performance of our method is comparable to state-of-the-art methods while maintaining its scalability advantages. We also applied our method in molecular optimization.