# Multi-Fidelity Bayesian Optimization With Across-Task Transferable Max-Value Entropy Search

arXiv: https://arxiv.org/abs/2403.09570 · HTML: https://mlsanity.com/paper/2403.09570/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 72 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning](https://mlsanity.com/paper/1903.04703/) (2019)
- [PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees](https://mlsanity.com/paper/2002.05551/) (2020)
- [Multi-Information Source Optimization](https://mlsanity.com/paper/1603.00389/) (2016)
- [Sionna RT: Differentiable Ray Tracing for Radio Propagation Modeling](https://mlsanity.com/paper/2303.11103/) (2023)
- [Functionality-preserving Black-box Optimization of Adversarial Windows Malware](https://mlsanity.com/paper/2003.13526/) (2020)
- [Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning](https://mlsanity.com/paper/2206.03996/) (2022)
- [GIBBON: General-purpose Information-Based Bayesian OptimisatioN](https://mlsanity.com/paper/2102.03324/) (2021)
- [Bayesian Optimization with High-Dimensional Outputs](https://mlsanity.com/paper/2106.12997/) (2021)
- [BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimization](https://mlsanity.com/paper/2002.01953/) (2020)
- [MUMBO: MUlti-task Max-value Bayesian Optimization](https://mlsanity.com/paper/2006.12093/) (2020)
- [Multi-Fidelity Bayesian Optimization with Unreliable Information Sources](https://mlsanity.com/paper/2210.13937/) (2022)
- [A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systems](https://mlsanity.com/paper/2212.01351/) (2022)
- [Optimization as Estimation with Gaussian Processes in Bandit Settings](https://mlsanity.com/paper/1510.06423/) (2015)
- [Guaranteed bounds on the Kullback-Leibler divergence of univariate mixtures using piecewise log-sum-exp inequalities](https://mlsanity.com/paper/1606.05850/) (2016)
- [Meta-Learning Priors for Safe Bayesian Optimization](https://mlsanity.com/paper/2210.00762/) (2022)
- [Calibrating Wireless Ray Tracing for Digital Twinning using Local Phase Error Estimates](https://mlsanity.com/paper/2312.12625/) (2023)
- [Bayesian Optimization with Formal Safety Guarantees via Online Conformal Prediction](https://mlsanity.com/paper/2306.17815/) (2023)
- [Bayesian and Multi-Armed Contextual Meta-Optimization for Efficient Wireless Radio Resource Management](https://mlsanity.com/paper/2301.06510/) (2023)
- [Tight Regret Bounds for Noisy Optimization of a Brownian Motion](https://mlsanity.com/paper/2001.09327/) (2020)
- [Lifelong Bayesian Optimization](https://mlsanity.com/paper/1905.12280/) (2019)

## Built on by (most cited first, 3 of 3)

- [Robust Bayesian Optimization via Localized Online Conformal Prediction](https://mlsanity.com/paper/2411.17387/) (2024)
- [Combinatorial Bandit Bayesian Optimization for Tensor Outputs](https://mlsanity.com/paper/2602.00640/) (2026)
- [Out-Of-The-Loop Multi-Fidelity Bayesian Optimization](https://mlsanity.com/paper/2608.04113/) (2026)

## Abstract (arXiv)

In many applications, ranging from logistics to engineering, a designer is faced with a sequence of optimization tasks for which the objectives are in the form of black-box functions that are costly to evaluate. Furthermore, higher-fidelity evaluations of the optimization objectives often entail a larger cost. Existing multi-fidelity black-box optimization strategies select candidate solutions and fidelity levels with the goal of maximizing the information about the optimal value or the optimal solution for the current task. Assuming that successive optimization tasks are related, this paper introduces a novel information-theoretic acquisition function that balances the need to acquire information about the current task with the goal of collecting information transferable to future tasks. The proposed method transfers across tasks distributions over parameters of a Gaussian process surrogate model by implementing particle-based variational Bayesian updates. Theoretical insights based on the analysis of the expected regret substantiate the benefits of acquiring transferable knowledge across tasks. Furthermore, experimental results across synthetic and real-world examples reveal that the proposed acquisition strategy that caters to future tasks can significantly improve the optimization efficiency as soon as a sufficient number of tasks is processed.