# A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes

arXiv: https://arxiv.org/abs/1811.00755 · HTML: https://mlsanity.com/paper/1811.00755/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2018.

## Built on by (most cited first, 30 of 52)

- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO](https://mlsanity.com/paper/2109.06716/) (2021)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Automated Reinforcement Learning (AutoRL): A Survey and Open Problems](https://mlsanity.com/paper/2201.03916/) (2022)
- [Joint Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/2210.02905/) (2022)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Looping in the Human Collaborative and Explainable Bayesian Optimization](https://mlsanity.com/paper/2310.17273/) (2023)
- [Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs](https://mlsanity.com/paper/2111.06537/) (2021)
- [Multi-Fidelity Bayesian Optimization with Unreliable Information Sources](https://mlsanity.com/paper/2210.13937/) (2022)
- [Discovering Diverse Athletic Jumping Strategies](https://mlsanity.com/paper/2105.00371/) (2021)
- [Multi-Fidelity Active Learning with GFlowNets](https://mlsanity.com/paper/2306.11715/) (2023)
- [Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2110.06980/) (2021)
- [Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling](https://mlsanity.com/paper/2402.18846/) (2024)
- [A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling](https://mlsanity.com/paper/2310.03298/) (2023)
- [A Lipschitz Bandits Approach for Continuous Hyperparameter Optimization](https://mlsanity.com/paper/2302.01539/) (2023)
- [Multifidelity Simulation-based Inference for Computationally Expensive Simulators](https://mlsanity.com/paper/2502.08416/) (2025)
- [Residual Gaussian Process: A Tractable Nonparametric Bayesian Emulator for Multi-fidelity Simulations](https://mlsanity.com/paper/2104.03743/) (2021)
- [Non-smooth Bayesian Optimization in Tuning Problems](https://mlsanity.com/paper/2109.07563/) (2021)
- [Movement Penalized Bayesian Optimization with Application to Wind Energy Systems](https://mlsanity.com/paper/2210.08087/) (2022)
- [Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks](https://mlsanity.com/paper/2106.09884/) (2021)
- [Efficient Hyperparameter Optimization for Physics-based Character Animation](https://mlsanity.com/paper/2104.12365/) (2021)
- [ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization](https://mlsanity.com/paper/2508.11551/) (2025)
- [HyperJump: Accelerating HyperBand via Risk Modelling](https://mlsanity.com/paper/2108.02479/) (2021)
- [Multi-fidelity Monte Carlo: a pseudo-marginal approach](https://mlsanity.com/paper/2210.01534/) (2022)
- [Robust Bayesian Target Value Optimization](https://mlsanity.com/paper/2301.04344/) (2023)
- [GAR: Generalized Autoregression for Multi-Fidelity Fusion](https://mlsanity.com/paper/2301.05729/) (2023)
- [Certified Multi-Fidelity Zeroth-Order Optimization](https://mlsanity.com/paper/2308.00978/) (2023)
- [Aligned Multi Objective Optimization](https://mlsanity.com/paper/2502.14096/) (2025)
- [A New Knowledge Gradient-based Method for Constrained Bayesian Optimization](https://mlsanity.com/paper/2101.08743/) (2021)
- [Scalable Interactive Machine Learning for Future Command and Control](https://mlsanity.com/paper/2402.06501/) (2024)

## Abstract (arXiv)

How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs? For example, when optimizing a robotic system, intelligently trading off computer simulations and real robot testings can lead to significant savings. Existing methods, such as multi-fidelity GP-UCB or Entropy Search-based approaches, either make simplistic assumptions on the interaction among different fidelities or use simple heuristics that lack theoretical guarantees. In this paper, we study multi-fidelity Bayesian optimization with complex structural dependencies among multiple outputs, and propose MF-MI-Greedy, a principled algorithmic framework for addressing this problem. In particular, we model different fidelities using additive Gaussian processes based on shared latent structures with the target function. Then we use cost-sensitive mutual information gain for efficient Bayesian global optimization. We propose a simple notion of regret which incorporates the cost of different fidelities, and prove that MF-MI-Greedy achieves low regret. We demonstrate the strong empirical performance of our algorithm on both synthetic and real-world datasets.