Data to 7 Oct 2026

BOHB: Robust and Efficient Hyperparameter Optimization at Scale

arXiv 1807.01774 ↗First posted 4 Jul 2018cs.LG stat.MLcited by 466

Not on the map. Why?

Abstract (arXiv)

Modern deep learning methods are very sensitive to many hyperparameters, and, due to the long training times of state-of-the-art models, vanilla Bayesian hyperparameter optimization is typically computationally infeasible. On the other hand, bandit-based configuration evaluation approaches based on random search lack guidance and do not converge to the best configurations as quickly. Here, we propose to combine the benefits of both Bayesian optimization and bandit-based methods, in order to achieve the best of both worlds: strong anytime performance and fast convergence to optimal configurations. We propose a new practical state-of-the-art hyperparameter optimization method, which consistently outperforms both Bayesian optimization and Hyperband on a wide range of problem types, including high-dimensional toy functions, support vector machines, feed-forward neural networks, Bayesian neural networks, deep reinforcement learning, and convolutional neural networks. Our method is robust and versatile, while at the same time being conceptually simple and easy to implement.

Key facts. Cited by 466 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 98; 2022: 89; 2023: 73; 2024: 70; 2025: 66; 2026: 70.

466 papers.

Sort:

By area:

2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
2023Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performancecited by 143
2022ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Modelscited by 141
2023Neural Architecture Search: Insights from 1000 Paperscited by 136
2024Large Language Models to Enhance Bayesian Optimizationcited by 127
2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88

38 more

2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
2021YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimizationcited by 61
2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
2022FederatedScope: A Flexible Federated Learning Platform for Heterogeneitycited by 53
2021Optimal Order Simple Regret for Gaussian Process Banditscited by 52
2021OpenBox: A Generalized Black-box Optimization Servicecited by 46
2021There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledgecited by 44
2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
2022Automated Dynamic Algorithm Configurationcited by 37
2022Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overviewcited by 37
2023Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networkscited by 37
2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
2025MAT-Agent: Adaptive Multi-Agent Training Optimizationcited by 34
2021Rapid Neural Architecture Search by Learning to Generate Graphs from Datasetscited by 33
2021KNAS: Green Neural Architecture Searchcited by 33
2021NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Taskscited by 32
2021AutoLoss: Automated Loss Function Search in Recommendationscited by 31
2023PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learningcited by 31
2023DisWOT: Student Architecture Search for Distillation WithOut Trainingcited by 29
2023Transfer Learning for Bayesian Optimization: A Surveycited by 27
2021BORE: Bayesian Optimization by Density-Ratio Estimationcited by 26
2022Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognitioncited by 26
2024Which LLM to Play? Convergence-Aware Online Model Selection with Time-Increasing Banditscited by 26
2024In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimizationcited by 26
2021VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decompositioncited by 25
2021Towards Green Automated Machine Learning: Status Quo and Future Directionscited by 25
2021EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Searchcited by 24
2021A survey on multi-objective hyperparameter optimization algorithms for Machine Learningcited by 24
2021Bayesian Optimization over Hybrid Spacescited by 23
2025Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewardscited by 23
2021Automated Discovery of Adaptive Attacks on Adversarial Defensescited by 22
2022AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuningcited by 22
2023Bilevel Fast Scene Adaptation for Low-Light Image Enhancementcited by 22
2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
2021Automatic Termination for Hyperparameter Optimizationcited by 21
2025Optuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?cited by 21
2021ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxiescited by 20

50 most cited of 466 All 466 (JSON)