mlsanityData to 7 Oct 2026

Multi-Fidelity Methods for Optimization: A Survey

arXiv 2402.09638 ↗First posted 15 Feb 2024cs.LG cs.NEcited by 5

Not on the map. Why?

Abstract (arXiv)

Real-world black-box optimization often involves time-consuming or costly experiments and simulations. Multi-fidelity optimization (MFO) stands out as a cost-effective strategy that balances high-fidelity accuracy with computational efficiency through a hierarchical fidelity approach. This survey presents a systematic exploration of MFO, underpinned by a novel text mining framework based on a pre-trained language model. We delve deep into the foundational principles and methodologies of MFO, focusing on three core components -- multi-fidelity surrogate models, fidelity management strategies, and optimization techniques. Additionally, this survey highlights the diverse applications of MFO across several key domains, including machine learning, engineering design optimization, and scientific discovery, showcasing the adaptability and effectiveness of MFO in tackling complex computational challenges. Furthermore, we also envision several emerging challenges and prospects in the MFO landscape, spanning scalability, the composition of lower fidelities, and the integration of human-in-the-loop approaches at the algorithmic level. We also address critical issues related to benchmarking and the advancement of open science within the MFO community. Overall, this survey aims to catalyze further research and foster collaborations in MFO, setting the stage for future innovations and breakthroughs in the field.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 69 of its 328 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

69 of 328 references.

References · 69

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  4. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  5. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  6. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  7. 2017Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equationscited by 362
  8. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  9. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 219
  10. 2018A System for Massively Parallel Hyperparameter Tuningcited by 201
  11. 2017Doubly Stochastic Variational Inference for Deep Gaussian Processescited by 143
  12. 2018Survey of multifidelity methods in uncertainty propagation, inference, and optimizationcited by 141
38 more
  1. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 131
  2. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 120
  3. 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
  4. 2015Batch Bayesian Optimization via Local Penalizationcited by 103
  5. 2016Multi-Information Source Optimizationcited by 83
  6. 2020Transfer learning based multi-fidelity physics informed deep neural networkcited by 68
  7. 2017Hyperparameter Importance Across Datasetscited by 62
  8. 2021YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimizationcited by 61
  9. 2019Deep Gaussian Processes for Multi-fidelity Modelingcited by 48
  10. 2021GIBBON: General-purpose Information-Based Bayesian OptimisatioNcited by 47
  11. 2020Multi-Fidelity Bayesian Optimization via Deep Neural Networkscited by 46
  12. 2020Cost-aware Bayesian Optimizationcited by 41
  13. 2021Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusioncited by 38
  14. 2016Multi-fidelity Gaussian Process Bandit Optimisationcited by 31
  15. 2020Overview of Gaussian process based multi-fidelity techniques with variable relationship between fidelitiescited by 24
  16. 2020MUMBO: MUlti-task Max-value Bayesian Optimizationcited by 21
  17. 2020MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurementscited by 18
  18. 2022Multi-Fidelity Bayesian Optimization with Unreliable Information Sourcescited by 17
  19. 2022Combining Multi-Fidelity Modelling and Asynchronous Batch Bayesian Optimizationcited by 16
  20. 2023InstOptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operatorscited by 16
  21. 2020BiLO-CPDP: Bi-Level Programming for Automated Model Discovery in Cross-Project Defect Predictioncited by 15
  22. 2021Scaling Gaussian Processes with Derivative Information Using Variational Inferencecited by 15
  23. 2022Supervising the Multi-Fidelity Race of Hyperparameter Configurationscited by 15
  24. 2023Multi-Fidelity Active Learning with GFlowNetscited by 15
  25. 2021Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directionscited by 14
  26. 2017Two-Archive Evolutionary Algorithm for Constrained Multi-Objective Optimizationcited by 13
  27. 2022Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scalecited by 13
  28. 2022PASHA: Efficient HPO and NAS with Progressive Resource Allocationcited by 13
  29. 2018Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learningcited by 12
  30. 2018The Gaussian Process Autoregressive Regression Model (GPAR)cited by 12
  31. 2019A support vector regression-based multi-fidelity surrogate modelcited by 12
  32. 2020DeepSQLi: Deep Semantic Learning for Testing SQL Injectioncited by 11
  33. 2016Dynamic Multi-Objectives Optimization with a Changing Number of Objectivescited by 7
  34. 2022A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Studycited by 7
  35. 2022Analytical Benchmark Problems for Multifidelity Optimization Methodscited by 7
  36. 2022Infinite-Fidelity Coregionalization for Physical Simulationcited by 7
  37. 2019Does Preference Always Help? A Holistic Study on Preference-Based Evolutionary Multi-Objective Optimisation Using Reference Pointscited by 6
  38. 2021Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networkscited by 6

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2024Fast Benchmarking of Asynchronous Multi-Fidelity Optimization on Zero-Cost Benchmarkscited by 2
  2. 2025Tree-Structured Parzen Estimator Can Solve Black-Box Combinatorial Optimization More Efficientlycited by 0
  3. 2026Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Datacited by 0
  4. 2026A Consistency-Centric Approach to Set-Based Optimization with Multiple Models of Unranked Fidelitycited by 0
  5. 2026BiFE: Search-Efficient Discovery of CPU-Only Branching Policies via LLM-based Bi-Fidelity Evolutioncited by 0