mlsanityData to 7 Oct 2026

No-Regret Bayesian Optimization with Unknown Hyperparameters

arXiv 1901.03357 ↗First posted 10 Jan 2019stat.ML cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) based on Gaussian process models is a powerful paradigm to optimize black-box functions that are expensive to evaluate. While several BO algorithms provably converge to the global optimum of the unknown function, they assume that the hyperparameters of the kernel are known in advance. This is not the case in practice and misspecification often causes these algorithms to converge to poor local optima. In this paper, we present the first BO algorithm that is provably no-regret and converges to the optimum without knowledge of the hyperparameters. During optimization we slowly adapt the hyperparameters of stationary kernels and thereby expand the associated function class over time, so that the BO algorithm considers more complex function candidates. Based on the theoretical insights, we propose several practical algorithms that achieve the empirical sample efficiency of BO with online hyperparameter estimation, but retain theoretical convergence guarantees. We evaluate our method on several benchmark problems.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 9; 2023: 9; 2024: 8; 2025: 13; 2026: 5.

50 papers.

Sort:

By area:

  1. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  2. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  3. 2021Misspecified Gaussian Process Bandit Optimizationcited by 52
  4. 2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 51
  5. 2022Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterizationcited by 35
  6. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  7. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  8. 2021Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applicationscited by 17
  9. 2022GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systemscited by 17
  10. 2023Posterior Sampling-Based Bayesian Optimization with Tighter Bayesian Regret Boundscited by 15
  11. 2024On Safety in Safe Bayesian Optimizationcited by 15
  12. 2022Regret Bounds for Expected Improvement Algorithms in Gaussian Process Bandit Optimizationcited by 14
38 more
  1. 2025Improved Regret Bounds for Gaussian Process Upper Confidence Bound in Bayesian Optimizationcited by 13
  2. 2022Scaling Gaussian Process Optimization by Evaluating a Few Unique Candidates Multiple Timescited by 12
  3. 2021Online Calibrated and Conformal Prediction Improves Bayesian Optimizationcited by 10
  4. 2025Safe exploration in reproducing kernel Hilbert spacescited by 10
  5. 2023Tuning Legged Locomotion Controllers via Safe Bayesian Optimizationcited by 8
  6. 2022Tuning Particle Accelerators with Safety Constraints using Bayesian Optimizationcited by 7
  7. 2023Batch Bayesian Optimization for Replicable Experimental Designcited by 7
  8. 2024Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimalcited by 6
  9. 2025DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Taskscited by 6
  10. 2025Thompson Sampling via Fine-Tuning of LLMscited by 6
  11. 2022Meta-Learning Hypothesis Spaces for Sequential Decision-makingcited by 5
  12. 2022Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimizationcited by 5
  13. 2023Sharp Calibrated Gaussian Processescited by 5
  14. 2023Adaptation to Misspecified Kernel Regularity in Kernelised Banditscited by 5
  15. 2021End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimizationcited by 4
  16. 2022Lifelong Bandit Optimization: No Prior and No Regretcited by 3
  17. 2024A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Liftingcited by 3
  18. 2024No-Regret Learning of Nash Equilibrium for Black-Box Games via Gaussian Processescited by 3
  19. 2024BO4IO: A Bayesian optimization approach to inverse optimization with uncertainty quantificationcited by 3
  20. 2025Every Call is Precious: Global Optimization of Black-Box Functions with Unknown Lipschitz Constantscited by 3
  21. 2023Provably Efficient Bayesian Optimization with Unknown Gaussian Process Hyperparameter Estimationcited by 2
  22. 2025Distance-Aware Error for Spline Networks: A Bottom-Up Approach to Uncertaintycited by 2
  23. 2025Reliable algorithm selection for machine learning-guided designcited by 2
  24. 2025Wasserstein Barycenter Gaussian Process based Bayesian Optimizationcited by 2
  25. 2025Just One Layer Norm Guarantees Stable Extrapolationcited by 2
  26. 2025Informed Initialization for Bayesian Optimization and Active Learningcited by 2
  27. 2022Safe and Adaptive Decision-Making for Optimization of Safety-Critical Systems: The ARTEO Algorithmcited by 1
  28. 2024Graph Agnostic Causal Bayesian Optimisationcited by 1
  29. 2025Adaptive Prior Selection in Gaussian Process Bandits with Thompson Samplingcited by 1
  30. 2025High-dimensional Nonparametric Contextual Bandit Problemcited by 1
  31. 2026Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputscited by 1
  32. 2023Generalization within in silico screeningcited by 0
  33. 2024Vector Optimization with Gaussian Process Banditscited by 0
  34. 2025Direct Regret Optimization in Bayesian Optimizationcited by 0
  35. 2026ORTHOBO: Orthogonal Bayesian Hyperparameter Optimizationcited by 0
  36. 2026Online Sharp-Calibrated Bayesian Optimizationcited by 0
  37. 2026Optimal Transport-based Permutation-Invariant Bayesian Optimization of Offshore Wind Farm Layoutscited by 0
  38. 2026Prequential E-Values for Selected-GP Near-Optimality Certificatescited by 0