# Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization

arXiv: https://arxiv.org/abs/2511.02570 · HTML: https://mlsanity.com/paper/2511.02570/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 85 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://mlsanity.com/paper/1312.3005/) (2013)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020](https://mlsanity.com/paper/2104.10201/) (2021)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO](https://mlsanity.com/paper/2109.06716/) (2021)
- [HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation](https://mlsanity.com/paper/2012.03826/) (2020)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [Hyperparameter Importance Across Datasets](https://mlsanity.com/paper/1710.04725/) (2017)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization](https://mlsanity.com/paper/2109.03670/) (2021)
- [Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning](https://mlsanity.com/paper/1909.12552/) (2019)
- [Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology](https://mlsanity.com/paper/2003.05155/) (2020)
- [Explaining Hyperparameter Optimization via Partial Dependence Plots](https://mlsanity.com/paper/2111.04820/) (2021)
- [Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2402.02746/) (2024)
- [PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning](https://mlsanity.com/paper/2306.12370/) (2023)
- [Looping in the Human Collaborative and Explainable Bayesian Optimization](https://mlsanity.com/paper/2310.17273/) (2023)
- [Practical Transfer Learning for Bayesian Optimization](https://mlsanity.com/paper/1802.02219/) (2018)
- [In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization](https://mlsanity.com/paper/2404.16795/) (2024)
- [Towards Green Automated Machine Learning: Status Quo and Future Directions](https://mlsanity.com/paper/2111.05850/) (2021)
- [Incorporating Expert Prior in Bayesian Optimisation via Space Warping](https://mlsanity.com/paper/2003.12250/) (2020)
- [A General Framework for User-Guided Bayesian Optimization](https://mlsanity.com/paper/2311.14645/) (2023)
- [Automatic Termination for Hyperparameter Optimization](https://mlsanity.com/paper/2104.08166/) (2021)
- [Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How](https://mlsanity.com/paper/2306.03828/) (2023)
- [DeepCAVE: An Interactive Analysis Tool for Automated Machine Learning](https://mlsanity.com/paper/2206.03493/) (2022)
- [Position: A Call to Action for a Human-Centered AutoML Paradigm](https://mlsanity.com/paper/2406.03348/) (2024)

## Built on by (most cited first, 2 of 2)

- [Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch](https://mlsanity.com/paper/2608.00316/) (2026)
- [Bayesian Optimization with Rich Auxiliary Information via LLMs](https://mlsanity.com/paper/2609.19437/) (2026)

## Abstract (arXiv)

Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO). However, most existing HPO methods only incorporate expert knowledge during initialization, limiting practitioners' ability to influence the optimization process as new insights emerge. This limits the applicability of BO in iterative machine learning development workflows. We propose DynaBO, a BO framework that enables continuous user control of the optimization process. Over time, DynaBO leverages provided user priors by augmenting the acquisition function with decaying, prior-weighted preferences while preserving asymptotic convergence guarantees. To enhance robustness, we introduce a surrogate-model-based safeguard that detects and, possibly, rejects misleading priors. We prove theoretical results on near-certain convergence, robustness to deceptive priors, and accelerated convergence when informative priors are provided. Extensive experiments across various HPO benchmarks show that DynaBO consistently outperforms state-of-the-art competitors across all benchmarks and for all prior kinds. Our results demonstrate that DynaBO enables reliable and efficient collaborative BO, bridging automated and manually controlled model development.