# High Dimensional Bayesian Optimization Assisted by Principal Component Analysis

arXiv: https://arxiv.org/abs/2007.00925 · HTML: https://mlsanity.com/paper/2007.00925/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB](https://mlsanity.com/paper/2303.00890/) (2023)
- [High Dimensional Bayesian Optimization with Kernel Principal Component Analysis](https://mlsanity.com/paper/2204.13753/) (2022)
- [OPT-GAN: A Broad-Spectrum Global Optimizer for Black-box Problems by Learning Distribution](https://mlsanity.com/paper/2102.03888/) (2021)
- [Bayesian Optimization with Informative Covariance](https://mlsanity.com/paper/2208.02704/) (2022)
- [LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions](https://mlsanity.com/paper/2311.11328/) (2023)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [High-Dimensional Bayesian Optimisation with Large-Scale Constraints -- An Application to Aeroelastic Tailoring](https://mlsanity.com/paper/2312.08891/) (2023)
- [Combining additivity and active subspaces for high-dimensional Gaussian process modeling](https://mlsanity.com/paper/2402.03809/) (2024)
- [Latent Space Arc Therapy Optimization](https://mlsanity.com/paper/2106.05846/) (2021)
- [Multi-view Bayesian optimisation in an input-output reduced space for engineering design](https://mlsanity.com/paper/2501.01552/) (2025)
- [GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization](https://mlsanity.com/paper/2601.18585/) (2026)
- [High-dimensional Multi-objective Bayesian Optimization with Learned Variable Interactions](https://mlsanity.com/paper/2608.11713/) (2026)
- [Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation](https://mlsanity.com/paper/2609.13396/) (2026)

## Abstract (arXiv)

Bayesian Optimization (BO) is a surrogate-assisted global optimization technique that has been successfully applied in various fields, e.g., automated machine learning and design optimization. Built upon a so-called infill-criterion and Gaussian Process regression (GPR), the BO technique suffers from a substantial computational complexity and hampered convergence rate as the dimension of the search spaces increases. Scaling up BO for high-dimensional optimization problems remains a challenging task. In this paper, we propose to tackle the scalability of BO by hybridizing it with a Principal Component Analysis (PCA), resulting in a novel PCA-assisted BO (PCA-BO) algorithm. Specifically, the PCA procedure learns a linear transformation from all the evaluated points during the run and selects dimensions in the transformed space according to the variability of evaluated points. We then construct the GPR model, and the infill-criterion in the space spanned by the selected dimensions. We assess the performance of our PCA-BO in terms of the empirical convergence rate and CPU time on multi-modal problems from the COCO benchmark framework. The experimental results show that PCA-BO can effectively reduce the CPU time incurred on high-dimensional problems, and maintains the convergence rate on problems with an adequate global structure. PCA-BO therefore provides a satisfactory trade-off between the convergence rate and computational efficiency opening new ways to benefit from the strength of BO approaches in high dimensional numerical optimization.