# A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences

arXiv: https://arxiv.org/abs/2406.04739 · HTML: https://mlsanity.com/paper/2406.04739/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 95 of its 157 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 30 of 95)

- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [Protein Design with Guided Discrete Diffusion](https://mlsanity.com/paper/2305.20009/) (2023)
- [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization](https://mlsanity.com/paper/2206.12411/) (2022)
- [Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020](https://mlsanity.com/paper/2104.10201/) (2021)
- [High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces](https://mlsanity.com/paper/2103.00349/) (2021)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining](https://mlsanity.com/paper/2006.09191/) (2020)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders](https://mlsanity.com/paper/2203.12742/) (2022)
- [Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2001.11659/) (2020)
- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces](https://mlsanity.com/paper/2109.10964/) (2021)
- [HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation](https://mlsanity.com/paper/2012.03826/) (2020)
- [Mat'ern Gaussian processes on Riemannian manifolds](https://mlsanity.com/paper/2006.10160/) (2020)
- [Combinatorial Bayesian Optimization using the Graph Cartesian Product](https://mlsanity.com/paper/1902.00448/) (2019)
- [Local Latent Space Bayesian Optimization over Structured Inputs](https://mlsanity.com/paper/2201.11872/) (2022)
- [A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization](https://mlsanity.com/paper/2111.05040/) (2021)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces](https://mlsanity.com/paper/1902.03229/) (2019)
- [Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1706.03673/) (2017)
- [BOCK : Bayesian Optimization with Cylindrical Kernels](https://mlsanity.com/paper/1806.01619/) (2018)
- [Bayesian Optimization of Combinatorial Structures](https://mlsanity.com/paper/1806.08838/) (2018)
- [Bayesian Optimization with Gradients](https://mlsanity.com/paper/1703.04389/) (2017)

## Built on by (most cited first, 23 of 23)

- [Latent Bayesian Optimization via Autoregressive Normalizing Flows](https://mlsanity.com/paper/2504.14889/) (2025)
- [Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spaces](https://mlsanity.com/paper/2507.03910/) (2025)
- [Variational Search Distributions](https://mlsanity.com/paper/2409.06142/) (2024)
- [Generative Bayesian Optimization: Generative Models as Acquisition Functions](https://mlsanity.com/paper/2510.25240/) (2025)
- [Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial](https://mlsanity.com/paper/2604.01328/) (2026)
- [CleanSurvival: Automated data preprocessing for time-to-event models using reinforcement learning](https://mlsanity.com/paper/2502.03946/) (2025)
- [Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimization](https://mlsanity.com/paper/2510.01471/) (2025)
- [BO4Mob: Bayesian Optimization Benchmarks for High-Dimensional Urban Mobility Problem](https://mlsanity.com/paper/2510.18824/) (2025)
- [Informed Initialization for Bayesian Optimization and Active Learning](https://mlsanity.com/paper/2510.23681/) (2025)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [Time-Aware Latent Space Bayesian Optimization](https://mlsanity.com/paper/2603.00935/) (2026)
- [AutoScout: Structured Optimization for Automating ML System Configuration](https://mlsanity.com/paper/2603.11603/) (2026)
- [Covering Multiple Objectives with a Small Set of Solutions Using Bayesian Optimization](https://mlsanity.com/paper/2501.19342/) (2025)
- [BOOST: A Data-Driven Framework for the Automated Joint Selection of Kernel and Acquisition Functions in Bayesian Optimization](https://mlsanity.com/paper/2508.02332/) (2025)
- [Information-Theoretic Bayesian Optimization for Bilevel Optimization Problems](https://mlsanity.com/paper/2509.21725/) (2025)
- [PepCompass: Navigating peptide embedding spaces using Riemannian Geometry](https://mlsanity.com/paper/2510.01988/) (2025)
- [MolLIBRA: Genetic Molecular Optimization with Multi-Fingerprint Surrogates and Text-Molecule Aligned Critic](https://mlsanity.com/paper/2602.07002/) (2026)
- [Information Theoretic Bayesian Optimization over the Probability Simplex](https://mlsanity.com/paper/2603.09793/) (2026)
- [BOAT: Navigating the Sea of In Silico Predictors for Antibody Design via Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2604.13980/) (2026)
- [Constraint-Aware Discrete Black-Box Optimization Using Tensor Decomposition](https://mlsanity.com/paper/2609.09370/) (2026)
- [Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation](https://mlsanity.com/paper/2609.13396/) (2026)
- [Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Search](https://mlsanity.com/paper/2609.34281/) (2026)
- [Atoms to Processes: The Role of Artificial Intelligence and Machine Learning in Chemical Engineering](https://mlsanity.com/paper/2610.02014/) (2026)

## Abstract (arXiv)

Optimizing discrete black-box functions is key in several domains, e.g. protein engineering and drug design. Due to the lack of gradient information and the need for sample efficiency, Bayesian optimization is an ideal candidate for these tasks. Several methods for high-dimensional continuous and categorical Bayesian optimization have been proposed recently. However, our survey of the field reveals highly heterogeneous experimental set-ups across methods and technical barriers for the replicability and application of published algorithms to real-world tasks. To address these issues, we develop a unified framework to test a vast array of high-dimensional Bayesian optimization methods and a collection of standardized black-box functions representing real-world application domains in chemistry and biology. These two components of the benchmark are each supported by flexible, scalable, and easily extendable software libraries (poli and poli-baselines), allowing practitioners to readily incorporate new optimization objectives or discrete optimizers. Project website: https://machinelearninglifescience.github.io/hdbo_benchmark