# Multi-Fidelity Bayesian Optimization via Deep Neural Networks

arXiv: https://arxiv.org/abs/2007.03117 · HTML: https://mlsanity.com/paper/2007.03117/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2020.

## Built on by (most cited first, 30 of 46)

- [Multi-Resolution Active Learning of Fourier Neural Operators](https://mlsanity.com/paper/2309.16971/) (2023)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Multi-Fidelity Bayesian Optimization with Unreliable Information Sources](https://mlsanity.com/paper/2210.13937/) (2022)
- [Scaling Laws for Hyperparameter Optimization](https://mlsanity.com/paper/2302.00441/) (2023)
- [Supervising the Multi-Fidelity Race of Hyperparameter Configurations](https://mlsanity.com/paper/2202.09774/) (2022)
- [Multi-fidelity Hierarchical Neural Processes](https://mlsanity.com/paper/2206.04872/) (2022)
- [Multi-Fidelity Active Learning with GFlowNets](https://mlsanity.com/paper/2306.11715/) (2023)
- [Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling](https://mlsanity.com/paper/2402.18846/) (2024)
- [Disentangled Multi-Fidelity Deep Bayesian Active Learning](https://mlsanity.com/paper/2305.04392/) (2023)
- [A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling](https://mlsanity.com/paper/2310.03298/) (2023)
- [Optimal Multi-Fidelity Best-Arm Identification](https://mlsanity.com/paper/2406.03033/) (2024)
- [Leveraging Trust for Joint Multi-Objective and Multi-Fidelity Optimization](https://mlsanity.com/paper/2112.13901/) (2021)
- [OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environments](https://mlsanity.com/paper/2312.12145/) (2023)
- [Best Practices for Multi-Fidelity Bayesian Optimization in Materials and Molecular Research](https://mlsanity.com/paper/2410.00544/) (2024)
- [Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks](https://mlsanity.com/paper/2106.09884/) (2021)
- [Multi-Fidelity Methods for Optimization: A Survey](https://mlsanity.com/paper/2402.09638/) (2024)
- [Improving Hyperparameter Optimization with Checkpointed Model Weights](https://mlsanity.com/paper/2406.18630/) (2024)
- [Amortized Auto-Tuning: Cost-Efficient Bayesian Transfer Optimization for Hyperparameter Recommendation](https://mlsanity.com/paper/2106.09179/) (2021)
- [Multi-fidelity Monte Carlo: a pseudo-marginal approach](https://mlsanity.com/paper/2210.01534/) (2022)
- [Practical multi-fidelity machine learning: fusion of deterministic and Bayesian models](https://mlsanity.com/paper/2407.15110/) (2024)
- [Cost-effective Framework for Gradual Domain Adaptation with Multifidelity](https://mlsanity.com/paper/2202.04359/) (2022)
- [Multifidelity Reinforcement Learning with Control Variates](https://mlsanity.com/paper/2206.05165/) (2022)
- [GAR: Generalized Autoregression for Multi-Fidelity Fusion](https://mlsanity.com/paper/2301.05729/) (2023)
- [Non-Myopic Multifidelity Bayesian Optimization](https://mlsanity.com/paper/2207.06325/) (2022)
- [Multi-Fidelity Covariance Estimation in the Log-Euclidean Geometry](https://mlsanity.com/paper/2301.13749/) (2023)
- [Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation](https://mlsanity.com/paper/2405.17918/) (2024)
- [Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors](https://mlsanity.com/paper/2602.00788/) (2026)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [On the Effects of Heterogeneous Errors on Multi-fidelity Bayesian Optimization](https://mlsanity.com/paper/2309.02771/) (2023)
- [Global Safe Sequential Learning via Efficient Knowledge Transfer](https://mlsanity.com/paper/2402.14402/) (2024)

## Abstract (arXiv)

Bayesian optimization (BO) is a popular framework to optimize black-box functions. In many applications, the objective function can be evaluated at multiple fidelities to enable a trade-off between the cost and accuracy. To reduce the optimization cost, many multi-fidelity BO methods have been proposed. Despite their success, these methods either ignore or over-simplify the strong, complex correlations across the fidelities, and hence can be inefficient in estimating the objective function. To address this issue, we propose Deep Neural Network Multi-Fidelity Bayesian Optimization (DNN-MFBO) that can flexibly capture all kinds of complicated relationships between the fidelities to improve the objective function estimation and hence the optimization performance. We use sequential, fidelity-wise Gauss-Hermite quadrature and moment-matching to fulfill a mutual information-based acquisition function, which is computationally tractable and efficient. We show the advantages of our method in both synthetic benchmark datasets and real-world applications in engineering design.