# Bayesian Optimization of Composite Functions

arXiv: https://arxiv.org/abs/1906.01537 · HTML: https://mlsanity.com/paper/1906.01537/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 30 of 46)

- [Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes](https://mlsanity.com/paper/2203.11382/) (2022)
- [Robust Multi-Objective Bayesian Optimization Under Input Noise](https://mlsanity.com/paper/2202.07549/) (2022)
- [Bayesian Optimization of Function Networks](https://mlsanity.com/paper/2112.15311/) (2021)
- [Bayesian Optimization with High-Dimensional Outputs](https://mlsanity.com/paper/2106.12997/) (2021)
- [Thinking inside the box: A tutorial on grey-box Bayesian optimization](https://mlsanity.com/paper/2201.00272/) (2022)
- [Bayesian Optimization over Hybrid Spaces](https://mlsanity.com/paper/2106.04682/) (2021)
- [Counterfactual Explanations for Arbitrary Regression Models](https://mlsanity.com/paper/2106.15212/) (2021)
- [Multi-Fidelity Cost-Aware Bayesian Optimization](https://mlsanity.com/paper/2211.02732/) (2022)
- [A General Recipe for Likelihood-free Bayesian Optimization](https://mlsanity.com/paper/2206.13035/) (2022)
- [Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structure](https://mlsanity.com/paper/2104.11667/) (2021)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Bayesian Optimization of Function Networks with Partial Evaluations](https://mlsanity.com/paper/2311.02146/) (2023)
- [Generative Adversarial Model-Based Optimization via Source Critic Regularization](https://mlsanity.com/paper/2402.06532/) (2024)
- [Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks](https://mlsanity.com/paper/2404.03099/) (2024)
- [Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimization](https://mlsanity.com/paper/2111.14911/) (2021)
- [No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations](https://mlsanity.com/paper/2305.03824/) (2023)
- [Joint Composite Latent Space Bayesian Optimization](https://mlsanity.com/paper/2311.02213/) (2023)
- [Pseudo-Bayesian Optimization](https://mlsanity.com/paper/2310.09766/) (2023)
- [Enhancing predictive capabilities in fusion burning plasmas through surrogate-based optimization in core transport solvers](https://mlsanity.com/paper/2312.12610/) (2023)
- [Regret Bounds for Noise-Free Cascaded Kernelized Bandits](https://mlsanity.com/paper/2211.05430/) (2022)
- [Robust Bayesian Target Value Optimization](https://mlsanity.com/paper/2301.04344/) (2023)
- [Bayesian Optimization for Function Compositions with Applications to Dynamic Pricing](https://mlsanity.com/paper/2303.11954/) (2023)
- [BOIS: Bayesian Optimization of Interconnected Systems](https://mlsanity.com/paper/2311.11254/) (2023)
- [Causal Bayesian Optimization via Exogenous Distribution Learning](https://mlsanity.com/paper/2402.02277/) (2024)
- [Knowledge-aware Evolutionary Graph Neural Architecture Search](https://mlsanity.com/paper/2411.17339/) (2024)
- [BILBO: BILevel Bayesian Optimization](https://mlsanity.com/paper/2502.02121/) (2025)
- [Multi-Objective Bayesian Optimization for Networked Black-Box Systems: A Path to Greener Profits and Smarter Designs](https://mlsanity.com/paper/2502.14121/) (2025)
- [Validation of Composite Systems by Discrepancy Propagation](https://mlsanity.com/paper/2210.12061/) (2022)
- [Scalable Bayesian optimization with high-dimensional outputs using randomized prior networks](https://mlsanity.com/paper/2302.07260/) (2023)
- [Bayesian Optimization of Expensive Nested Grey-Box Functions](https://mlsanity.com/paper/2306.05150/) (2023)

## Abstract (arXiv)

We consider optimization of composite objective functions, i.e., of the form f(x)=g(h(x)), where h is a black-box derivative-free expensive-to-evaluate function with vector-valued outputs, and g is a cheap-to-evaluate real-valued function. While these problems can be solved with standard Bayesian optimization, we propose a novel approach that exploits the composite structure of the objective function to substantially improve sampling efficiency. Our approach models h using a multi-output Gaussian process and chooses where to sample using the expected improvement evaluated on the implied non-Gaussian posterior on f, which we call expected improvement for composite functions (ei). Although ei cannot be computed in closed form, we provide a novel stochastic gradient estimator that allows its efficient maximization. We also show that our approach is asymptotically consistent, i.e., that it recovers a globally optimal solution as sampling effort grows to infinity, generalizing previous convergence results for classical expected improvement. Numerical experiments show that our approach dramatically outperforms standard Bayesian optimization benchmarks, reducing simple regret by several orders of magnitude.