Targeted Variance Reduction: Robust Bayesian Optimization of Black-Box Simulators with Noise Parameters

The optimization of a black-box simulator over control parameters x arises in a myriad of scientific applications. In such applications, the simulator often takes the form f(x,boldsymbolθ), where boldsymbolθ are parameters that are uncertain in practice. Robust optimization aims to optimize the objective mathbbE[f(x,boldsymbolTheta)], where boldsymbolTheta ~ P is a random variable that models uncertainty on boldsymbolθ. For this, existing black-box methods typically employ a two-stage approach for selecting the next point (x,boldsymbolθ), where x and boldsymbolθ are optimized separately via different acquisition functions. As such, these approaches do not employ a joint acquisition over (x,boldsymbolθ), and thus may fail to fully exploit control-to-noise interactions for effective robust optimization. To address this, we propose a new Bayesian optimization method called Targeted Variance Reduction (TVR). The TVR leverages a novel joint acquisition function over (x,boldsymbolθ), which targets variance reduction on the objective within the desired region of improvement. Under a Gaussian process surrogate on f, the TVR acquisition can be evaluated in closed form, and reveals an insightful exploration-exploitation-precision trade-off for robust black-box optimization. The TVR can further accommodate a broad class of non-Gaussian distributions on P via a careful integration of normalizing flows. We demonstrate the improved performance of TVR over the state-of-the-art in a suite of numerical experiments and an application to the robust design of automobile brake discs under operational uncertainty.