# Distributionally Robust Bayesian Optimization with varphi-divergences

arXiv: https://arxiv.org/abs/2203.02128 · HTML: https://mlsanity.com/paper/2203.02128/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 66 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 30 of 31)

- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach](https://mlsanity.com/paper/1610.03425/) (2016)
- [Variance-based regularization with convex objectives](https://mlsanity.com/paper/1610.02581/) (2016)
- [Distributionally Robust Logistic Regression](https://mlsanity.com/paper/1509.09259/) (2015)
- [Regularization via Mass Transportation](https://mlsanity.com/paper/1710.10016/) (2017)
- [Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space](https://mlsanity.com/paper/1706.01825/) (2017)
- [Distributionally Robust Optimization and Generalization in Kernel Methods](https://mlsanity.com/paper/1905.10943/) (2019)
- [Coping with Label Shift via Distributionally Robust Optimisation](https://mlsanity.com/paper/2010.12230/) (2020)
- [Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits](https://mlsanity.com/paper/2002.02518/) (2020)
- [Adversarially Robust Optimization with Gaussian Processes](https://mlsanity.com/paper/1810.10775/) (2018)
- [Distributionally Robust Bayesian Optimization](https://mlsanity.com/paper/2002.09038/) (2020)
- [Bayesian Optimisation over Multiple Continuous and Categorical Inputs](https://mlsanity.com/paper/1906.08878/) (2019)
- [Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis](https://mlsanity.com/paper/2110.12459/) (2021)
- [Practical Bayesian optimization in the presence of outliers](https://mlsanity.com/paper/1712.04567/) (2017)
- [Wasserstein Distributionally Robust Optimization and Variation Regularization](https://mlsanity.com/paper/1712.06050/) (2017)
- [Robust Reinforcement Learning with Wasserstein Constraint](https://mlsanity.com/paper/2006.00945/) (2020)
- [Knowing The What But Not The Where in Bayesian Optimization](https://mlsanity.com/paper/1905.02685/) (2019)
- [Unscented Bayesian Optimization for Safe Robot Grasping](https://mlsanity.com/paper/1603.02038/) (2016)
- [Bayesian optimisation under uncertain inputs](https://mlsanity.com/paper/1902.07908/) (2019)
- [No-Regret Learning in Unknown Games with Correlated Payoffs](https://mlsanity.com/paper/1909.08540/) (2019)
- [Distributional Robustness with IPMs and links to Regularization and GANs](https://mlsanity.com/paper/2006.04349/) (2020)
- [Bayesian Distributionally Robust Optimization](https://mlsanity.com/paper/2112.08625/) (2021)
- [Distributionally Robust Bayesian Quadrature Optimization](https://mlsanity.com/paper/2001.06814/) (2020)
- [Accelerating Experimental Design by Incorporating Experimenter Hunches](https://mlsanity.com/paper/1907.09065/) (2019)
- [Generalised Lipschitz Regularisation Equals Distributional Robustness](https://mlsanity.com/paper/2002.04197/) (2020)
- [Amazon SageMaker Automatic Model Tuning: Scalable Gradient-Free Optimization](https://mlsanity.com/paper/2012.08489/) (2020)
- [Learning and Decision-Making with Data: Optimal Formulations and Phase Transitions](https://mlsanity.com/paper/2109.06911/) (2021)
- [Lipschitz Networks and Distributional Robustness](https://mlsanity.com/paper/1809.01129/) (2018)
- [The Inductive Bias of Restricted f-GANs](https://mlsanity.com/paper/1809.04542/) (2018)

## Built on by (most cited first, 16 of 16)

- [Offline-to-online hyperparameter transfer for stochastic bandits](https://mlsanity.com/paper/2501.02926/) (2025)
- [High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategy](https://mlsanity.com/paper/2402.03104/) (2024)
- [Robust Entropy Search for Safe Efficient Bayesian Optimization](https://mlsanity.com/paper/2405.19059/) (2024)
- [High Dimensional Bayesian Optimization using Lasso Variable Selection](https://mlsanity.com/paper/2504.01743/) (2025)
- [Regret Analysis of Posterior Sampling-Based Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2507.09828/) (2025)
- [Distributionally Robust Optimization via Diffusion Ambiguity Modeling](https://mlsanity.com/paper/2510.22757/) (2025)
- [Wasserstein Distributionally Robust Bayesian Optimization with Continuous Context](https://mlsanity.com/paper/2503.20341/) (2025)
- [Distributionally Robust Optimization via Generative Ambiguity Modeling](https://mlsanity.com/paper/2602.08976/) (2026)
- [Decision Making under Model Misspecification: DRO with Robust Bayesian Ambiguity Sets](https://mlsanity.com/paper/2505.03585/) (2025)
- [Learning Relevant Contextual Variables Within Bayesian Optimization](https://mlsanity.com/paper/2305.14120/) (2023)
- [Bayesian Optimization for General Reaction Conditions](https://mlsanity.com/paper/2502.18966/) (2025)
- [3D-Learning: Diffusion-Augmented Distributionally Robust Decision-Focused Learning](https://mlsanity.com/paper/2602.02943/) (2026)
- [Ensemble Distributionally Robust Bayesian Optimisation with Continuous Context](https://mlsanity.com/paper/2605.07565/) (2026)
- [Conservative neural posterior estimation via distributionally robust training](https://mlsanity.com/paper/2605.28516/) (2026)
- [Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption](https://mlsanity.com/paper/2605.30089/) (2026)
- [Robust Bayesian Decision Making under Adversarial Uncertainty](https://mlsanity.com/paper/2607.08590/) (2026)

## Abstract (arXiv)

The study of robustness has received much attention due to its inevitability in data-driven settings where many systems face uncertainty. One such example of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted, yet there only exists a limited number of works dedicated to this direction. In particular, there is the work of Kirschner et al. (2020), which bridges the existing literature of Distributionally Robust Optimization (DRO) by casting the BO problem from the lens of DRO. While this work is pioneering, it admittedly suffers from various practical shortcomings such as finite contexts assumptions, leaving behind the main question Can one devise a computationally tractable algorithm for solving this DRO-BO problem? In this work, we tackle this question to a large degree of generality by considering robustness against data-shift in varphi-divergences, which subsumes many popular choices, such as the chi2-divergence, Total Variation, and the extant Kullback-Leibler (KL) divergence. We show that the DRO-BO problem in this setting is equivalent to a finite-dimensional optimization problem which, even in the continuous context setting, can be easily implemented with provable sublinear regret bounds. We then show experimentally that our method surpasses existing methods, attesting to the theoretical results.