mlsanityData to 7 Oct 2026

Distributionally Robust Bayesian Optimization with varphi-divergences

arXiv 2203.02128 ↗First posted 4 Mar 2022cs.LG math.OC stat.MLcited by 16

Not on the map. Why?

Abstract (arXiv)

The study of robustness has received much attention due to its inevitability in data-driven settings where many systems face uncertainty. One such example of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted, yet there only exists a limited number of works dedicated to this direction. In particular, there is the work of Kirschner et al. (2020), which bridges the existing literature of Distributionally Robust Optimization (DRO) by casting the BO problem from the lens of DRO. While this work is pioneering, it admittedly suffers from various practical shortcomings such as finite contexts assumptions, leaving behind the main question Can one devise a computationally tractable algorithm for solving this DRO-BO problem? In this work, we tackle this question to a large degree of generality by considering robustness against data-shift in varphi-divergences, which subsumes many popular choices, such as the chi2-divergence, Total Variation, and the extant Kullback-Leibler (KL) divergence. We show that the DRO-BO problem in this setting is equivalent to a finite-dimensional optimization problem which, even in the continuous context setting, can be easily implemented with provable sublinear regret bounds. We then show experimentally that our method surpasses existing methods, attesting to the theoretical results.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 31 of its 66 references that are in the map. First posted on arXiv 4 Mar 2022.

Builds on

31 of 66 references.

References · 31

  1. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  2. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 202
  3. 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 182
  4. 2016Variance-based regularization with convex objectivescited by 153
  5. 2015Distributionally Robust Logistic Regressioncited by 124
  6. 2017Regularization via Mass Transportationcited by 79
  7. 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
  8. 2019Distributionally Robust Optimization and Generalization in Kernel Methodscited by 73
  9. 2020Coping with Label Shift via Distributionally Robust Optimisationcited by 69
  10. 2018Adversarially Robust Optimization with Gaussian Processescited by 61
  11. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 61
  12. 2020Distributionally Robust Bayesian Optimizationcited by 60
19 more
  1. 2019Bayesian Optimisation over Multiple Continuous and Categorical Inputscited by 55
  2. 2021Non-convex Distributionally Robust Optimization: Non-asymptotic Analysiscited by 39
  3. 2017Practical Bayesian optimization in the presence of outlierscited by 25
  4. 2017Wasserstein Distributionally Robust Optimization and Variation Regularizationcited by 25
  5. 2019Knowing The What But Not The Where in Bayesian Optimizationcited by 20
  6. 2016Unscented Bayesian Optimization for Safe Robot Graspingcited by 18
  7. 2019Bayesian optimisation under uncertain inputscited by 18
  8. 2019No-Regret Learning in Unknown Games with Correlated Payoffscited by 18
  9. 2020Robust Reinforcement Learning with Wasserstein Constraintcited by 18
  10. 2020Distributional Robustness with IPMs and links to Regularization and GANscited by 17
  11. 2021Bayesian Distributionally Robust Optimizationcited by 16
  12. 2020Distributionally Robust Bayesian Quadrature Optimizationcited by 15
  13. 2020Generalised Lipschitz Regularisation Equals Distributional Robustnesscited by 14
  14. 2019Accelerating Experimental Design by Incorporating Experimenter Hunchescited by 13
  15. 2020Amazon SageMaker Automatic Model Tuning: Scalable Gradient-Free Optimizationcited by 9
  16. 2021Learning and Decision-Making with Data: Optimal Formulations and Phase Transitionscited by 7
  17. 2018Lipschitz Networks and Distributional Robustnesscited by 6
  18. 2018The Inductive Bias of Restricted f-GANscited by 5
  19. 2020Gaussian Process Bandit Optimization of the Thermodynamic Variational Objectivecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 7; 2026: 6.

16 papers.

Sort:

By area:

  1. 2025Offline-to-online hyperparameter transfer for stochastic banditscited by 6
  2. 2024High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategycited by 5
  3. 2024Robust Entropy Search for Safe Efficient Bayesian Optimizationcited by 4
  4. 2025Distributionally Robust Optimization via Diffusion Ambiguity Modelingcited by 4
  5. 2025Wasserstein Distributionally Robust Bayesian Optimization with Continuous Contextcited by 3
  6. 2025High Dimensional Bayesian Optimization using Lasso Variable Selectioncited by 3
  7. 2025Regret Analysis of Posterior Sampling-Based Expected Improvement for Bayesian Optimizationcited by 3
  8. 2026Distributionally Robust Optimization via Generative Ambiguity Modelingcited by 3
  9. 2025Decision Making under Model Misspecification: DRO with Robust Bayesian Ambiguity Setscited by 2
  10. 2023Learning Relevant Contextual Variables Within Bayesian Optimizationcited by 0
  11. 2025Bayesian Optimization for General Reaction Conditionscited by 0
  12. 20263D-Learning: Diffusion-Augmented Distributionally Robust Decision-Focused Learningcited by 0
4 more
  1. 2026Ensemble Distributionally Robust Bayesian Optimisation with Continuous Contextcited by 0
  2. 2026Conservative neural posterior estimation via distributionally robust trainingcited by 0
  3. 2026Distributionally Robust Set Representation Learning Under Inference-Time Element Corruptioncited by 0
  4. 2026Robust Bayesian Decision Making under Adversarial Uncertaintycited by 0