# On Provably Robust Meta-Bayesian Optimization

arXiv: https://arxiv.org/abs/2206.06872 · HTML: https://mlsanity.com/paper/2206.06872/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [On Kernelized Multi-armed Bandits](https://mlsanity.com/paper/1704.00445/) (2017)
- [Meta-Learning: A Survey](https://mlsanity.com/paper/1810.03548/) (2018)
- [Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/1805.09801/) (2018)
- [Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning](https://mlsanity.com/paper/1903.04703/) (2019)
- [Multi-Information Source Optimization](https://mlsanity.com/paper/1603.00389/) (2016)
- [Federated Bayesian Optimization via Thompson Sampling](https://mlsanity.com/paper/2010.10154/) (2020)
- [Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization](https://mlsanity.com/paper/1904.02642/) (2019)
- [Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning](https://mlsanity.com/paper/1909.12552/) (2019)
- [Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks](https://mlsanity.com/paper/1903.12489/) (2019)
- [Differentially Private Federated Bayesian Optimization with Distributed Exploration](https://mlsanity.com/paper/2110.14153/) (2021)
- [Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior](https://mlsanity.com/paper/1811.09558/) (2018)
- [Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning](https://mlsanity.com/paper/1806.04798/) (2018)
- [Warm Starting Bayesian Optimization](https://mlsanity.com/paper/1608.03585/) (2016)
- [No-Regret Learning in Unknown Games with Correlated Payoffs](https://mlsanity.com/paper/1909.08540/) (2019)
- [Bayesian Optimization with Binary Auxiliary Information](https://mlsanity.com/paper/1906.07277/) (2019)
- [R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games](https://mlsanity.com/paper/2006.16679/) (2020)

## Built on by (most cited first, 8 of 8)

- [Sample-Then-Optimize Batch Neural Thompson Sampling](https://mlsanity.com/paper/2210.06850/) (2022)
- [Bayesian Optimization under Stochastic Delayed Feedback](https://mlsanity.com/paper/2206.09341/) (2022)
- [Training-Free Neural Active Learning with Initialization-Robustness Guarantees](https://mlsanity.com/paper/2306.04454/) (2023)
- [Scalable Meta-Learning with Gaussian Processes](https://mlsanity.com/paper/2312.00742/) (2023)
- [Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces](https://mlsanity.com/paper/2409.05325/) (2024)
- [Clustering-based Meta Bayesian Optimization with Theoretical Guarantee](https://mlsanity.com/paper/2503.06093/) (2025)
- [SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2601.22131/) (2026)
- [Regime-Conditioned Evaluation in Multi-Context Bayesian Optimization](https://mlsanity.com/paper/2605.04895/) (2026)

## Abstract (arXiv)

Bayesian optimization (BO) has become popular for sequential optimization of black-box functions. When BO is used to optimize a target function, we often have access to previous evaluations of potentially related functions. This begs the question as to whether we can leverage these previous experiences to accelerate the current BO task through meta-learning (meta-BO), while ensuring robustness against potentially harmful dissimilar tasks that could sabotage the convergence of BO. This paper introduces two scalable and provably robust meta-BO algorithms: robust meta-Gaussian process-upper confidence bound (RM-GP-UCB) and RM-GP-Thompson sampling (RM-GP-TS). We prove that both algorithms are asymptotically no-regret even when some or all previous tasks are dissimilar to the current task, and show that RM-GP-UCB enjoys a better theoretical robustness than RM-GP-TS. We also exploit the theoretical guarantees to optimize the weights assigned to individual previous tasks through regret minimization via online learning, which diminishes the impact of dissimilar tasks and hence further enhances the robustness. Empirical evaluations show that (a) RM-GP-UCB performs effectively and consistently across various applications, and (b) RM-GP-TS, despite being less robust than RM-GP-UCB both in theory and in practice, performs competitively in some scenarios with less dissimilar tasks and is more computationally efficient.