Data to 7 Oct 2026

Bayesian Optimization in a Billion Dimensions via Random Embeddings

arXiv 1301.1942 ↗First posted 9 Jan 2013stat.ML cs.LGcited by 178

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high-dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple, has important invariance properties, and applies to domains with both categorical and continuous variables. We present a thorough theoretical analysis of REMBO. Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low. They also show that REMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver.

Key facts. Cited by 178 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jan 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 24; 2023: 28; 2024: 27; 2025: 41; 2026: 29.

178 papers.

Sort:

By area:

2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 99
2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
2022Recent Advances in Bayesian Optimizationcited by 74
2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 70
2023InstructZero: Efficient Instruction Optimization for Black-Box Large Language Modelscited by 68
2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54

38 more

2022BBTv2: Towards a Gradient-Free Future with Large Language Modelscited by 53
2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 51
2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 50
2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
2022Automated Dynamic Algorithm Configurationcited by 37
2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 33
2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 29
2021TREGO: a Trust-Region Framework for Efficient Global Optimizationcited by 28
2023Gradient-Free Textual Inversioncited by 28
2023Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorialcited by 27
2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
2023Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddingscited by 24
2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
2025Understanding High-Dimensional Bayesian Optimizationcited by 19
2023Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOBcited by 18
2023Active Learning and Bayesian Optimization: a Unified Perspective to Learn with a Goalcited by 17
2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
2024Offline Multi-Objective Optimizationcited by 14
2021Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selectioncited by 12
2023Efficient Federated Prompt Tuning for Black-box Large Pre-trained Modelscited by 12
2023Multi-fidelity Bayesian Optimization: A Reviewcited by 12
2025Posterior Inference with Diffusion Models for High-dimensional Black-box Optimizationcited by 12
2021Revisiting Bayesian Optimization in the light of the COCO benchmarkcited by 10
2024Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimizationcited by 9
2024Hyperband-based Bayesian Optimization for Black-box Prompt Selectioncited by 9
2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 9
2023Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimizationcited by 8
2023Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimationcited by 8
2021Direct-Search for a Class of Stochastic Min-Max Problemscited by 7
2021Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezingcited by 7
2021Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiationcited by 7
2022High Dimensional Bayesian Optimization with Kernel Principal Component Analysiscited by 7
2022Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approachcited by 7
2021Global optimization using random embeddingscited by 6
2021A comparison of mixed-variables Bayesian optimization approachescited by 6
2021BoGraph: Structured Bayesian Optimization From Logs for Expensive Systems with Many Parameterscited by 6

50 most cited of 178 All 178 (JSON)