# Dimensionality Reduction Techniques for Global Bayesian Optimisation

arXiv: https://arxiv.org/abs/2412.09183 · HTML: https://mlsanity.com/paper/2412.09183/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing](https://mlsanity.com/paper/1903.10145/) (2019)
- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning](https://mlsanity.com/paper/2106.03609/) (2021)
- [A Domain-Shrinking based Bayesian Optimization Algorithm with Order-Optimal Regret Performance](https://mlsanity.com/paper/2010.13997/) (2020)
- [BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization](https://mlsanity.com/paper/1910.06403/) (2019)
- [TVAE: Triplet-Based Variational Autoencoder using Metric Learning](https://mlsanity.com/paper/1802.04403/) (2018)
- [A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality](https://mlsanity.com/paper/2003.09673/) (2020)
- [Global optimization using random embeddings](https://mlsanity.com/paper/2107.12102/) (2021)

## Built on by (most cited first, 1 of 1)

- [Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization](https://mlsanity.com/paper/2510.15435/) (2025)

## Abstract (arXiv)

Bayesian Optimisation (BO) is a state-of-the-art global optimisation technique for black-box problems where derivative information is unavailable, and sample efficiency is crucial. However, improving the general scalability of BO has proved challenging. Here, we explore Latent Space Bayesian Optimisation (LSBO), that applies dimensionality reduction to perform BO in a reduced-dimensional subspace. While early LSBO methods used (linear) random projections (Wang et al., 2013), we employ Variational Autoencoders (VAEs) to manage more complex data structures and general DR tasks. Building on Grosnit et. al. (2021), we analyse the VAE-based LSBO framework, focusing on VAE retraining and deep metric loss. We suggest a few key corrections in their implementation, originally designed for tasks such as molecule generation, and reformulate the algorithm for broader optimisation purposes. Our numerical results show that structured latent manifolds improve BO performance. Additionally, we examine the use of the Mat'ern-frac52 kernel for Gaussian Processes in this LSBO context. We also integrate Sequential Domain Reduction (SDR), a standard global optimization efficiency strategy, into BO. SDR is included in a GPU-based environment using textitBoTorch, both in the original and VAE-generated latent spaces, marking the first application of SDR within LSBO.