mlsanityData to 14 Oct 2026

MOCA-HESP: Meta High-dimensional Bayesian Optimization for Combinatorial and Mixed Spaces via Hyper-ellipsoid Partitioning

arXiv 2508.06847 ↗First posted 9 Aug 2025stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

High-dimensional Bayesian Optimization (BO) has attracted significant attention in recent research. However, existing methods have mainly focused on optimizing in continuous domains, while combinatorial (ordinal and categorical) and mixed domains still remain challenging. In this paper, we first propose MOCA-HESP, a novel high-dimensional BO method for combinatorial and mixed variables. The key idea is to leverage the hyper-ellipsoid space partitioning (HESP) technique with different categorical encoders to work with high-dimensional, combinatorial and mixed spaces, while adaptively selecting the optimal encoders for HESP using a multi-armed bandit technique. Our method, MOCA-HESP, is designed as a textitmeta-algorithm such that it can incorporate other combinatorial and mixed BO optimizers to further enhance the optimizers' performance. Finally, we develop three practical BO methods by integrating MOCA-HESP with state-of-the-art BO optimizers for combinatorial and mixed variables: standard BO, CASMOPOLITAN, and Bounce. Our experimental results on various synthetic and real-world benchmarks show that our methods outperform existing baselines. Our code implementation can be found at https://github.com/LamNgo1/moca-hesp

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 9 Aug 2025.

Builds on

29 of 49 references.

References · 29

  1. 2016The CMA Evolution Strategy: A Tutorialcited by 377
  2. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  3. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  4. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 92
  5. 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 89
  6. 2019Combinatorial Bayesian Optimization using the Graph Cartesian Productcited by 79
  7. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 66
  8. 2018Bayesian Optimization of Combinatorial Structurescited by 64
  9. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  10. 2019Bayesian Optimisation over Multiple Continuous and Categorical Inputscited by 56
  11. 2020BOSS: Bayesian Optimization over String Spacescited by 54
  12. 2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 53
17 more
  1. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 52
  2. 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
  3. 2022Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterizationcited by 37
  4. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 36
  5. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 28
  6. 2023Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddingscited by 24
  7. 2021Bayesian Optimization over Hybrid Spacescited by 23
  8. 2021ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxiescited by 22
  9. 2020Optimizing Coverage and Capacity in Cellular Networks using Machine Learningcited by 21
  10. 2022AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisationcited by 16
  11. 2023Framework and Benchmarks for Combinatorial and Mixed-variable Bayesian Optimizationcited by 16
  12. 2022Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spacescited by 14
  13. 2024Pareto Front-Diverse Batch Multi-Objective Bayesian Optimizationcited by 13
  14. 2015Low Autocorrelation Binary Sequencescited by 11
  15. 2022CMA-ES with Margin: Lower-Bounding Marginal Probability for Mixed-Integer Black-Box Optimizationcited by 6
  16. 2024High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategycited by 6
  17. 2024BOIDS: High-dimensional Bayesian Optimization via Incumbent-guided Direction Lines and Subspace Embeddingscited by 4

Built on by

Not cited yet.