# Additive Tree-Structured Covariance Function for Conditional Parameter Spaces in Bayesian Optimization

arXiv: https://arxiv.org/abs/2006.11771 · HTML: https://mlsanity.com/paper/2006.11771/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2020.

## Built on by (most cited first, 6 of 6)

- [Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces](https://mlsanity.com/paper/2207.00879/) (2022)
- [Monte Carlo Tree Search based Space Transfer for Black-box Optimization](https://mlsanity.com/paper/2412.07186/) (2024)
- [Amortized Auto-Tuning: Cost-Efficient Bayesian Transfer Optimization for Hyperparameter Recommendation](https://mlsanity.com/paper/2106.09179/) (2021)
- [Log-Linear-Time Gaussian Processes Using Binary Tree Kernels](https://mlsanity.com/paper/2210.01633/) (2022)
- [Modeling All Response Surfaces in One for Conditional Search Spaces](https://mlsanity.com/paper/2501.04260/) (2025)
- [Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces](https://mlsanity.com/paper/2601.20800/) (2026)

## Abstract (arXiv)

Bayesian optimization (BO) is a sample-efficient global optimization algorithm for black-box functions which are expensive to evaluate. Existing literature on model based optimization in conditional parameter spaces are usually built on trees. In this work, we generalize the additive assumption to tree-structured functions and propose an additive tree-structured covariance function, showing improved sample-efficiency, wider applicability and greater flexibility. Furthermore, by incorporating the structure information of parameter spaces and the additive assumption in the BO loop, we develop a parallel algorithm to optimize the acquisition function and this optimization can be performed in a low dimensional space. We demonstrate our method on an optimization benchmark function, as well as on a neural network model compression problem, and experimental results show our approach significantly outperforms the current state of the art for conditional parameter optimization including SMAC, TPE and Jenatton et al. (2017).