mlsanityData to 7 Oct 2026

Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization

arXiv 2210.01628 ↗First posted 4 Oct 2022cs.LG cs.AIcited by 37

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is a class of popular methods for expensive black-box optimization, and has been widely applied to many scenarios. However, BO suffers from the curse of dimensionality, and scaling it to high-dimensional problems is still a challenge. In this paper, we propose a variable selection method MCTS-VS based on Monte Carlo tree search (MCTS), to iteratively select and optimize a subset of variables. That is, MCTS-VS constructs a low-dimensional subspace via MCTS and optimizes in the subspace with any BO algorithm. We give a theoretical analysis of the general variable selection method to reveal how it can work. Experiments on high-dimensional synthetic functions and real-world problems (i.e., NAS-bench problems and MuJoCo locomotion tasks) show that MCTS-VS equipped with a proper BO optimizer can achieve state-of-the-art performance.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 25 of its 54 references that are in the map. First posted on arXiv 4 Oct 2022.

Builds on

25 of 54 references.

References · 25

  1. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  2. 2018A Tutorial on Bayesian Optimizationcited by 734
  3. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  4. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  5. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  6. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  7. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  8. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  9. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  10. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  11. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  12. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
13 more
  1. 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 87
  2. 2021A survey on high-dimensional Gaussian process modeling with application to Bayesian optimizationcited by 76
  3. 2020NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Searchcited by 64
  4. 2017The reparameterization trick for acquisition functionscited by 61
  5. 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 58
  6. 2021TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Searchcited by 57
  7. 2018High Dimensional Bayesian Optimization Using Dropoutcited by 40
  8. 2017On the choice of the low-dimensional domain for global optimization via random embeddingscited by 25
  9. 2017Decentralized High-Dimensional Bayesian Optimization with Factor Graphscited by 23
  10. 2020High-Dimensional Bayesian Optimization via Tree-Structured Additive Modelscited by 22
  11. 2014A warped kernel improving robustness in Bayesian optimization via random embeddingscited by 17
  12. 2021Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selectioncited by 12
  13. 2019Bayesian Optimization using Pseudo-Pointscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 11; 2025: 11; 2026: 7.

37 papers.

Sort:

By area:

  1. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  2. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
  3. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  4. 2025Understanding High-Dimensional Bayesian Optimizationcited by 19
  5. 2024Offline Multi-Objective Optimizationcited by 14
  6. 2024Expensive Multi-Objective Bayesian Optimization Based on Diffusion Modelscited by 10
  7. 2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 9
  8. 2023Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimizationcited by 8
  9. 2023Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimationcited by 8
  10. 2024Monte Carlo Tree Search based Space Transfer for Black-box Optimizationcited by 7
  11. 2023High-dimensional Bayesian Optimization with Group Testingcited by 5
  12. 2023Joint Composite Latent Space Bayesian Optimizationcited by 5
25 more
  1. 2024High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategycited by 5
  2. 2023Racing Control Variable Genetic Programming for Symbolic Regressioncited by 4
  3. 2024Safe Bayesian Optimization for the Control of High-Dimensional Embodied Systemscited by 4
  4. 2025BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placementcited by 4
  5. 2023Bayesian Optimisation Against Climate Change: Applications and Benchmarkscited by 3
  6. 2023Stochastic Bayesian Optimization with Unknown Continuous Context Distribution via Kernel Density Estimationcited by 3
  7. 2024No-Regret Learning of Nash Equilibrium for Black-Box Games via Gaussian Processescited by 3
  8. 2025High Dimensional Bayesian Optimization using Lasso Variable Selectioncited by 3
  9. 2024Expected Coordinate Improvement for High-Dimensional Bayesian Optimizationcited by 2
  10. 2024Navigating in High-Dimensional Search Space: A Hierarchical Bayesian Optimization Approachcited by 2
  11. 2025Enhancing Trust-Region Bayesian Optimization via Newton Methodscited by 2
  12. 2025Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testingcited by 1
  13. 2026NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spacescited by 1
  14. 2026Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimizationcited by 1
  15. 2024Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Searchcited by 0
  16. 2025An Adaptive Dropout Approach for High-Dimensional Bayesian Optimizationcited by 0
  17. 2025BOOST: A Data-Driven Framework for the Automated Joint Selection of Kernel and Acquisition Functions in Bayesian Optimizationcited by 0
  18. 2025MOCA-HESP: Meta High-dimensional Bayesian Optimization for Combinatorial and Mixed Spaces via Hyper-ellipsoid Partitioningcited by 0
  19. 2025Towards safe control parameter tuning in distributed multi-agent systemscited by 0
  20. 2025Quantum feature encoding optimizationcited by 0
  21. 2026BONSAI: Bayesian Optimization with Natural Simplicity and Interpretabilitycited by 0
  22. 2026Rethinking Trust Region Bayesian Optimization in High Dimensionscited by 0
  23. 2026Automated Random Embedding for Practical Bayesian Optimization with Unknown Effective Dimensioncited by 0
  24. 2026Rethinking Learnability in Offline Data-driven Optimizationcited by 0
  25. 2026Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Searchcited by 0