# Good practices for Bayesian Optimization of high dimensional structured spaces

arXiv: https://arxiv.org/abs/2012.15471 · HTML: https://mlsanity.com/paper/2012.15471/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2020.

## Built on by (most cited first, 24 of 24)

- [Local Latent Space Bayesian Optimization over Structured Inputs](https://mlsanity.com/paper/2201.11872/) (2022)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning](https://mlsanity.com/paper/2106.03609/) (2021)
- [LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lasso](https://mlsanity.com/paper/2111.02790/) (2021)
- [Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?](https://mlsanity.com/paper/2301.12844/) (2023)
- [Discovering Many Diverse Solutions with Bayesian Optimization](https://mlsanity.com/paper/2210.10953/) (2022)
- [Latent Bayesian Optimization via Autoregressive Normalizing Flows](https://mlsanity.com/paper/2504.14889/) (2025)
- [Advancing Bayesian Optimization via Learning Correlated Latent Space](https://mlsanity.com/paper/2310.20258/) (2023)
- [PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance](https://mlsanity.com/paper/2312.16983/) (2023)
- [Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach](https://mlsanity.com/paper/2207.00128/) (2022)
- [Deep Kernel Methods Learn Better: From Cards to Process Optimization](https://mlsanity.com/paper/2303.14554/) (2023)
- [Inverse Design of Copolymers Including Stoichiometry and Chain Architecture](https://mlsanity.com/paper/2410.02824/) (2024)
- [Latent Space Bayesian Optimization with Latent Data Augmentation for Enhanced Exploration](https://mlsanity.com/paper/2302.02399/) (2023)
- [Joint Composite Latent Space Bayesian Optimization](https://mlsanity.com/paper/2311.02213/) (2023)
- [Co-orchestration of Multiple Instruments to Uncover Structure-Property Relationships in Combinatorial Libraries](https://mlsanity.com/paper/2402.02198/) (2024)
- [Safe Bayesian Optimization for the Control of High-Dimensional Embodied Systems](https://mlsanity.com/paper/2412.20350/) (2024)
- [Expected Coordinate Improvement for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2404.11917/) (2024)
- [A machine learning approach for fighting the curse of dimensionality in global optimization](https://mlsanity.com/paper/2110.14985/) (2021)
- [Large Scale Multi-Task Bayesian Optimization with Large Language Models](https://mlsanity.com/paper/2503.08131/) (2025)
- [Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spaces](https://mlsanity.com/paper/2202.00893/) (2022)
- [Covering Multiple Objectives with a Small Set of Solutions Using Bayesian Optimization](https://mlsanity.com/paper/2501.19342/) (2025)
- [Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization](https://mlsanity.com/paper/2510.15435/) (2025)
- [Mesh-Native Physics-Informed Graph Surrogates for TCAD-in-the-Loop Design Space Exploration](https://mlsanity.com/paper/2609.02988/) (2026)
- [BOReFT: Manifold Steering of Language Models for Black-box Optimization](https://mlsanity.com/paper/2609.33722/) (2026)

## Abstract (arXiv)

The increasing availability of structured but high dimensional data has opened new opportunities for optimization. One emerging and promising avenue is the exploration of unsupervised methods for projecting structured high dimensional data into low dimensional continuous representations, simplifying the optimization problem and enabling the application of traditional optimization methods. However, this line of research has been purely methodological with little connection to the needs of practitioners so far. In this paper, we study the effect of different search space design choices for performing Bayesian Optimization in high dimensional structured datasets. In particular, we analyse the influence of the dimensionality of the latent space, the role of the acquisition function and evaluate new methods to automatically define the optimization bounds in the latent space. Finally, based on experimental results using synthetic and real datasets, we provide recommendations for the practitioners.