mlsanityData to 7 Oct 2026

Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces

arXiv 2102.07188 ↗First posted 14 Feb 2021stat.ML cs.LGcited by 51

Not on the map. Why?

Abstract (arXiv)

High-dimensional black-box optimisation remains an important yet notoriously challenging problem. Despite the success of Bayesian optimisation methods on continuous domains, domains that are categorical, or that mix continuous and categorical variables, remain challenging. We propose a novel solution -- we combine local optimisation with a tailored kernel design, effectively handling high-dimensional categorical and mixed search spaces, whilst retaining sample efficiency. We further derive convergence guarantee for the proposed approach. Finally, we demonstrate empirically that our method outperforms the current baselines on a variety of synthetic and real-world tasks in terms of performance, computational costs, or both.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Builds on 31 of its 67 references that are in the map. First posted on arXiv 14 Feb 2021.

Builds on

31 of 67 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2018A Tutorial on Bayesian Optimizationcited by 734
  4. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  5. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  6. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  7. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  8. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  9. 2018AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networkscited by 137
  10. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  11. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  12. 2018GenAttack: Practical Black-box Attacks with Gradient-Free Optimizationcited by 82
19 more
  1. 2019Combinatorial Bayesian Optimization using the Graph Cartesian Productcited by 78
  2. 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
  3. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 65
  4. 2020Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attackscited by 65
  5. 2018Bayesian Optimization of Combinatorial Structurescited by 64
  6. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 61
  7. 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 58
  8. 2019Bayesian Optimisation over Multiple Continuous and Categorical Inputscited by 55
  9. 2017Batched High-dimensional Bayesian Optimization via Structural Kernel Learningcited by 51
  10. 2019No-Regret Bayesian Optimization with Unknown Hyperparameterscited by 50
  11. 2019Mixed-Variable Bayesian Optimizationcited by 29
  12. 2020Optimal Transport Kernels for Sequential and Parallel Neural Architecture Searchcited by 29
  13. 2019Bayesian Optimization for Categorical and Category-Specific Continuous Inputscited by 28
  14. 2017On the choice of the low-dimensional domain for global optimization via random embeddingscited by 25
  15. 2014A warped kernel improving robustness in Bayesian optimization via random embeddingscited by 17
  16. 2020Optimizing Discrete Spaces via Expensive Evaluations: A Learning to Search Frameworkcited by 11
  17. 2020Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraintscited by 9
  18. 2020Combinatorial Black-Box Optimization with Expert Advicecited by 8
  19. 2017Adaptive Restart of the Optimized Gradient Method for Convex Optimizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 13; 2024: 10; 2025: 13; 2026: 4.

51 papers.

Sort:

By area:

  1. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 144
  2. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  3. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  4. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
  5. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 50
  6. 2023AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuningcited by 49
  7. 2022Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterizationcited by 35
  8. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  9. 2022Local Bayesian optimization via maximizing probability of descentcited by 29
  10. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
  11. 2023Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddingscited by 24
  12. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
38 more
  1. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  2. 2022Bayesian Generational Population-Based Trainingcited by 20
  3. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 20
  4. 2021BOiLS: Bayesian Optimisation for Logic Synthesiscited by 18
  5. 2023Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruningcited by 17
  6. 2023Framework and Benchmarks for Combinatorial and Mixed-variable Bayesian Optimizationcited by 15
  7. 2022AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisationcited by 14
  8. 2024Improved Off-policy Reinforcement Learning in Biological Sequence Designcited by 13
  9. 2024E2ETune: End-to-End Knob Tuning via Fine-tuned Generative Language Modelcited by 9
  10. 2024Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimizationcited by 9
  11. 2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 9
  12. 2023SOBER: Highly Parallel Bayesian Optimization and Bayesian Quadrature over Discrete and Mixed Spacescited by 7
  13. 2023Bayesian Optimisation of Functions on Graphscited by 7
  14. 2024BruSLeAttack: A Query-Efficient Score-Based Black-Box Sparse Adversarial Attackcited by 7
  15. 2021Approximate Neural Architecture Search via Operation Distribution Learningcited by 6
  16. 2023Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approachcited by 6
  17. 2021Adversarial Attacks on Graph Classification via Bayesian Optimisationcited by 5
  18. 2024High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategycited by 5
  19. 2023Black-Box Optimization with Implicit Constraints for Public Policycited by 4
  20. 2024A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Liftingcited by 3
  21. 2025High Dimensional Bayesian Optimization using Lasso Variable Selectioncited by 3
  22. 2024CATBench: A Compiler Autotuning Benchmarking Suite for Black-box Optimizationcited by 2
  23. 2025Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimizationcited by 2
  24. 2024Bayesian Optimization of Functions over Node Subsets in Graphscited by 1
  25. 2025A Guide to Bayesian Optimization in Bioprocess Engineeringcited by 1
  26. 2025Omnipresent Yet Overlooked: Heat Kernels in Combinatorial Bayesian Optimizationcited by 1
  27. 2021Searching in the Forest for Local Bayesian Optimizationcited by 0
  28. 2022Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spacescited by 0
  29. 2025Selecting Critical Scenarios of DER Adoption in Distribution Grids Using Bayesian Optimizationcited by 0
  30. 2025Weighted Euclidean Distance Matrices over Mixed Continuous and Categorical Inputs for Gaussian Process Modelscited by 0
  31. 2025CatBOX: A Categorical-Continuous Bayesian Optimization with Spectral Mixture Kernels for Accelerated Catalysis Experimentscited by 0
  32. 2025MOCA-HESP: Meta High-dimensional Bayesian Optimization for Combinatorial and Mixed Spaces via Hyper-ellipsoid Partitioningcited by 0
  33. 2025COM-BOM: Bayesian Exemplar Search for Efficiently Exploring the Accuracy-Calibration Pareto Frontiercited by 0
  34. 2025A reproducible comparative study of categorical kernels for Gaussian process regression, with new clustering-based nested kernelscited by 0
  35. 2025Iterated Population Based Training with Task-Agnostic Restartscited by 0
  36. 2026Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processescited by 0
  37. 2026SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimizationcited by 0
  38. 2026Categorical Optimization with Bayesian Anchored Latent Trust Regions for Structural Design under High-Dimensional Uncertaintycited by 0

50 most cited of 51 All 51 (JSON)