# Targeted Variance Reduction: Robust Bayesian Optimization of Black-Box Simulators with Noise Parameters

arXiv: https://arxiv.org/abs/2403.03816 · HTML: https://mlsanity.com/paper/2403.03816/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Adversarially Robust Optimization with Gaussian Processes](https://mlsanity.com/paper/1810.10775/) (2018)
- [Local Gaussian process approximation for large computer experiments](https://mlsanity.com/paper/1303.0383/) (2013)
- [Bayesian Optimization with Expensive Integrands](https://mlsanity.com/paper/1803.08661/) (2018)
- [Bayesian Optimization for Synthetic Gene Design](https://mlsanity.com/paper/1505.01627/) (2015)
- [Unscented Bayesian Optimization for Safe Robot Grasping](https://mlsanity.com/paper/1603.02038/) (2016)
- [A hierarchical expected improvement method for Bayesian optimization](https://mlsanity.com/paper/1911.07285/) (2019)
- [Triangulation candidates for Bayesian optimization](https://mlsanity.com/paper/2112.07457/) (2021)
- [Function-on-function kriging, with applications to 3D printing of aortic tissues](https://mlsanity.com/paper/1910.01754/) (2019)
- [A graphical multi-fidelity Gaussian process model, with application to emulation of heavy-ion collisions](https://mlsanity.com/paper/2108.00306/) (2021)
- [Additive Multi-Index Gaussian process modeling, with application to multi-physics surrogate modeling of the quark-gluon plasma](https://mlsanity.com/paper/2306.07299/) (2023)
- [Robust expected improvement for Bayesian optimization](https://mlsanity.com/paper/2302.08612/) (2023)

## Built on by (most cited first, 2 of 2)

- [Expected Diverse Utility (EDU): Diverse Bayesian Optimization of Expensive Computer Simulators](https://mlsanity.com/paper/2410.01196/) (2024)
- [Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators](https://mlsanity.com/paper/2410.12690/) (2024)

## Abstract (arXiv)

The optimization of a black-box simulator over control parameters x arises in a myriad of scientific applications. In such applications, the simulator often takes the form f(x,boldsymbolθ), where boldsymbolθ are parameters that are uncertain in practice. Robust optimization aims to optimize the objective mathbbE[f(x,boldsymbolTheta)], where boldsymbolTheta ~ P is a random variable that models uncertainty on boldsymbolθ. For this, existing black-box methods typically employ a two-stage approach for selecting the next point (x,boldsymbolθ), where x and boldsymbolθ are optimized separately via different acquisition functions. As such, these approaches do not employ a joint acquisition over (x,boldsymbolθ), and thus may fail to fully exploit control-to-noise interactions for effective robust optimization. To address this, we propose a new Bayesian optimization method called Targeted Variance Reduction (TVR). The TVR leverages a novel joint acquisition function over (x,boldsymbolθ), which targets variance reduction on the objective within the desired region of improvement. Under a Gaussian process surrogate on f, the TVR acquisition can be evaluated in closed form, and reveals an insightful exploration-exploitation-precision trade-off for robust black-box optimization. The TVR can further accommodate a broad class of non-Gaussian distributions on P via a careful integration of normalizing flows. We demonstrate the improved performance of TVR over the state-of-the-art in a suite of numerical experiments and an application to the robust design of automobile brake discs under operational uncertainty.