FastBO: Fast HPO and NAS with Adaptive Fidelity Identification
Not on the map. Why?
Abstract (arXiv)
Hyperparameter optimization (HPO) and neural architecture search (NAS) are powerful in attaining state-of-the-art machine learning models, with Bayesian optimization (BO) standing out as a mainstream method. Extending BO into the multi-fidelity setting has been an emerging research topic, but faces the challenge of determining an appropriate fidelity for each hyperparameter configuration to fit the surrogate model. To tackle the challenge, we propose a multi-fidelity BO method named FastBO, which adaptively decides the fidelity for each configuration and efficiently offers strong performance. The advantages are achieved based on the novel concepts of efficient point and saturation point for each configuration.We also show that our adaptive fidelity identification strategy provides a way to extend any single-fidelity method to the multi-fidelity setting, highlighting its generality and applicability.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 1 Sep 2024.
Builds on
29 of 41 references.
References · 29
- 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
- 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
- 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
- 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
- 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 219
- 2018A System for Massively Parallel Hyperparameter Tuningcited by 201
- 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
- 2014Freeze-Thaw Bayesian Optimizationcited by 86
- 2022πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimizationcited by 69
- 2021The Shape of Learning Curves: a Reviewcited by 67
- 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 62
- 2017Bayesian Optimization with Gradientscited by 59
17 more
- 2019Tabular Benchmarks for Joint Architecture and Hyperparameter Optimizationcited by 57
- 2021Explaining Hyperparameter Optimization via Partial Dependence Plotscited by 35
- 2021Scaling Gaussian Processes with Derivative Information Using Variational Inferencecited by 15
- 2022Supervising the Multi-Fidelity Race of Hyperparameter Configurationscited by 15
- 2015Unbounded Bayesian Optimization via Regularizationcited by 13
- 2019Accelerating Experimental Design by Incorporating Experimenter Hunchescited by 13
- 2022Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scalecited by 13
- 2022PASHA: Efficient HPO and NAS with Progressive Resource Allocationcited by 13
- 2018Combination of Hyperband and Bayesian Optimization for Hyperparameter Optimization in Deep Learningcited by 12
- 2022Scalable First-Order Bayesian Optimization via Structured Automatic Differentiationcited by 10
- 2023Optimizing Hyperparameters with Conformal Quantile Regressioncited by 10
- 2024Regulating Model Reliance on Non-Robust Features by Smoothing Input Marginal Densitycited by 10
- 2022Fast Parallel Exact Inference on Bayesian Networks: Postercited by 3
- 2022Fast Parallel Bayesian Network Structure Learningcited by 3
- 2022Improving Accuracy of Interpretability Measures in Hyperparameter Optimization via Bayesian Algorithm Executioncited by 2
- 2024A Backdoor-based Explainable AI Benchmark for High Fidelity Evaluation of Attributionscited by 2
- 2024Fast-PGM: Fast Probabilistic Graphical Model Learning and Inferencecited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.