# Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior

arXiv: https://arxiv.org/abs/1811.09558 · HTML: https://mlsanity.com/paper/1811.09558/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2018.

## Built on by (most cited first, 30 of 31)

- [Towards Learning Universal Hyperparameter Optimizers with Transformers](https://mlsanity.com/paper/2205.13320/) (2022)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [Optimal Order Simple Regret for Gaussian Process Bandits](https://mlsanity.com/paper/2108.09262/) (2021)
- [Bayesian Optimization with Conformal Prediction Sets](https://mlsanity.com/paper/2210.12496/) (2022)
- [Transfer Learning for Bayesian Optimization: A Survey](https://mlsanity.com/paper/2302.05927/) (2023)
- [Posterior Sampling-Based Bayesian Optimization with Tighter Bayesian Regret Bounds](https://mlsanity.com/paper/2311.03760/) (2023)
- [Open Problem: Tight Online Confidence Intervals for RKHS Elements](https://mlsanity.com/paper/2110.15458/) (2021)
- [Batch Bayesian optimisation via density-ratio estimation with guarantees](https://mlsanity.com/paper/2209.10715/) (2022)
- [Transfer Learning for Bayesian Optimization on Heterogeneous Search Spaces](https://mlsanity.com/paper/2309.16597/) (2023)
- [On Provably Robust Meta-Bayesian Optimization](https://mlsanity.com/paper/2206.06872/) (2022)
- [Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal](https://mlsanity.com/paper/2410.10384/) (2024)
- [Offline-to-online hyperparameter transfer for stochastic bandits](https://mlsanity.com/paper/2501.02926/) (2025)
- [Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimization](https://mlsanity.com/paper/2205.04901/) (2022)
- [Collaborative Learning in Kernel-based Bandits for Distributed Users](https://mlsanity.com/paper/2207.07948/) (2022)
- [Meta-Learning Hypothesis Spaces for Sequential Decision-making](https://mlsanity.com/paper/2202.00602/) (2022)
- [Pre-training helps Bayesian optimization too](https://mlsanity.com/paper/2207.03084/) (2022)
- [A Scalable Test Problem Generator for Sequential Transfer Optimization](https://mlsanity.com/paper/2304.08503/) (2023)
- [HPC Storage Service Autotuning Using Variational-Autoencoder-Guided Asynchronous Bayesian Optimization](https://mlsanity.com/paper/2210.00798/) (2022)
- [Lifelong Bandit Optimization: No Prior and No Regret](https://mlsanity.com/paper/2210.15513/) (2022)
- [HyperBO+: Pre-training a universal prior for Bayesian optimization with hierarchical Gaussian processes](https://mlsanity.com/paper/2212.10538/) (2022)
- [Rover: An online Spark SQL tuning service via generalized transfer learning](https://mlsanity.com/paper/2302.04046/) (2023)
- [A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Lifting](https://mlsanity.com/paper/2404.12219/) (2024)
- [ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation](https://mlsanity.com/paper/2604.23099/) (2026)
- [An Adaptive Dimension Reduction Estimation Method for High-dimensional Bayesian Optimization](https://mlsanity.com/paper/2403.05425/) (2024)
- [Provable Accelerated Bayesian Optimization with Knowledge Transfer](https://mlsanity.com/paper/2511.03125/) (2025)
- [Robust Bayesian Optimisation with Unbounded Corruptions](https://mlsanity.com/paper/2511.15315/) (2025)
- [Generalization within in silico screening](https://mlsanity.com/paper/2307.09379/) (2023)
- [Few-shot Scooping Under Domain Shift via Simulated Maximal Deployment Gaps](https://mlsanity.com/paper/2408.02949/) (2024)
- [Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces](https://mlsanity.com/paper/2409.05325/) (2024)
- [Clustering-based Meta Bayesian Optimization with Theoretical Guarantee](https://mlsanity.com/paper/2503.06093/) (2025)

## Abstract (arXiv)

Bayesian optimization usually assumes that a Bayesian prior is given. However, the strong theoretical guarantees in Bayesian optimization are often regrettably compromised in practice because of unknown parameters in the prior. In this paper, we adopt a variant of empirical Bayes and show that, by estimating the Gaussian process prior from offline data sampled from the same prior and constructing unbiased estimators of the posterior, variants of both GP-UCB and probability of improvement achieve a near-zero regret bound, which decreases to a constant proportional to the observational noise as the number of offline data and the number of online evaluations increase. Empirically, we have verified our approach on challenging simulated robotic problems featuring task and motion planning.