A new integral loss function for Bayesian optimization

We consider the problem of maximizing a real-valued continuous function f using a Bayesian approach. Since the early work of Jonas Mockus and Antanas vZilinskas in the 70's, the problem of optimization is usually formulated by considering the loss function max f - Mn (where Mn denotes the best function value observed after n evaluations of f). This loss function puts emphasis on the value of the maximum, at the expense of the location of the maximizer. In the special case of a one-step Bayes-optimal strategy, it leads to the classical Expected Improvement (EI) sampling criterion. This is a special case of a Stepwise Uncertainty Reduction (SUR) strategy, where the risk associated to a certain uncertainty measure (here, the expected loss) on the quantity of interest is minimized at each step of the algorithm. In this article, assuming that f is defined over a measure space (mathbbX, λ), we propose to consider instead the integral loss function int_mathbbX (f - Mn)+, dλ, and we show that this leads, in the case of a Gaussian process prior, to a new numerically tractable sampling criterion that we call rm EI2 (for Expected Integrated Expected Improvement). A numerical experiment illustrates that a SUR strategy based on this new sampling criterion reduces the error on both the value and the location of the maximizer faster than the EI-based strategy.