mlsanityData to 14 Oct 2026

PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning

arXiv 2306.12370 ↗First posted 21 Jun 2023cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Hyperparameters of Deep Learning (DL) pipelines are crucial for their downstream performance. While a large number of methods for Hyperparameter Optimization (HPO) have been developed, their incurred costs are often untenable for modern DL. Consequently, manual experimentation is still the most prevalent approach to optimize hyperparameters, relying on the researcher's intuition, domain knowledge, and cheap preliminary explorations. To resolve this misalignment between HPO algorithms and DL researchers, we propose PriorBand, an HPO algorithm tailored to DL, able to utilize both expert beliefs and cheap proxy tasks. Empirically, we demonstrate PriorBand's efficiency across a range of DL benchmarks and show its gains under informative expert input and robustness against poor expert beliefs

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 33 of its 68 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

33 of 68 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  5. 2016Wide Residual Networkscited by 3,070
  6. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  7. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 471
  8. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  9. 2021ResNet strikes back: An improved training procedure in timmcited by 368
  10. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
  11. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 221
  12. 2018A System for Massively Parallel Hyperparameter Tuningcited by 202
21 more
  1. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  2. 2020Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learningcited by 174
  3. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 132
  4. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 121
  5. 2015Batch Bayesian Optimization via Local Penalizationcited by 104
  6. 2017On the State of the Art of Evaluation in Neural Language Modelscited by 104
  7. 2014Freeze-Thaw Bayesian Optimizationcited by 90
  8. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  9. 2021Few-Shot Bayesian Optimization with Deep Kernel Surrogatescited by 71
  10. 2022πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimizationcited by 70
  11. 2020Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDLcited by 67
  12. 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 65
  13. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  14. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  15. 2018Bayesian Optimization in AlphaGocited by 55
  16. 2019Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learningcited by 50
  17. 2018Practical Transfer Learning for Bayesian Optimizationcited by 26
  18. 2019Knowing The What But Not The Where in Bayesian Optimizationcited by 20
  19. 2019Bayesian Optimization for Iterative Learningcited by 19
  20. 2020Hyperparameter Transfer Across Developer Adjustmentscited by 11
  21. 2020AutoHAS: Efficient Hyperparameter and Architecture Searchcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 14; 2025: 13; 2026: 2.

31 papers.

Sort:

By area:

  1. 2024Large Language Model Agent for Hyper-Parameter Optimizationcited by 64
  2. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  3. 2024In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimizationcited by 26
  4. 2023A General Framework for User-Guided Bayesian Optimizationcited by 22
  5. 2024Position: A Call to Action for a Human-Centered AutoML Paradigmcited by 14
  6. 2024Benchmarking Deep Learning Models on NVIDIA Jetson Nano for Real-Time Systems: An Empirical Investigationcited by 9
  7. 2023Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learningcited by 8
  8. 2024Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimizationcited by 7
  9. 2024Don't Waste Your Time: Early Stopping Cross-Validationcited by 6
  10. 2024Dual-view X-ray Detection: Can AI Detect Prohibited Items from Dual-view X-ray Images like Humans?cited by 4
  11. 2025Hyperparameter Optimization via Interacting with Probabilistic Circuitscited by 4
  12. 2024Optimizing Time Series Forecasting Architectures: A Hierarchical Neural Architecture Search Approachcited by 3
19 more
  1. 2025Auto-nnU-Net: Towards Automated Medical Image Segmentationcited by 3
  2. 2025carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarkscited by 3
  3. 2025LLMs as In-Context Meta-Learners for Model and Hyperparameter Selectioncited by 3
  4. 2024LMEMs for post-hoc analysis of HPO Benchmarkingcited by 2
  5. 2024Towards certifiable AI in aviation: landscape, challenges, and opportunitiescited by 2
  6. 2025Frozen Layers: Memory-efficient Many-fidelity Hyperparameter Optimizationcited by 2
  7. 2025CoTune: Co-evolutionary Configuration Tuningcited by 2
  8. 2025Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuningcited by 2
  9. 2025Dynamic Priors in Bayesian Optimization for Hyperparameter Optimizationcited by 2
  10. 2025DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learningcited by 2
  11. 2025Multi-objective Hyperparameter Optimization in the Age of Deep Learningcited by 1
  12. 2024Strong convexity-guided hyper-parameter optimization for flatter lossescited by 0
  13. 2024Primitive Agentic First-Order Optimizationcited by 0
  14. 2024Bootstrap Sampling Rate Greater than 1.0 May Improve Random Forest Performancecited by 0
  15. 2025XAutoLM: Efficient Fine-Tuning of Language Models via Meta-Learning and AutoMLcited by 0
  16. 2025Quickly Tuning Foundation Models for Image Segmentationcited by 0
  17. 2025Iterated Population Based Training with Task-Agnostic Restartscited by 0
  18. 2026Regime-Conditioned Evaluation in Multi-Context Bayesian Optimizationcited by 0
  19. 2026Provably Reduced Sample Cost in Prior-Guided Hyperparameter Optimizationcited by 0