# What do you Mean? The Role of the Mean Function in Bayesian Optimisation

arXiv: https://arxiv.org/abs/2004.08349 · HTML: https://mlsanity.com/paper/2004.08349/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2020.

## Built on by (most cited first, 7 of 7)

- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [Self-Correcting Bayesian Optimization through Bayesian Active Learning](https://mlsanity.com/paper/2304.11005/) (2023)
- [Robust and Conjugate Gaussian Process Regression](https://mlsanity.com/paper/2311.00463/) (2023)
- [How Bayesian Should Bayesian Optimisation Be?](https://mlsanity.com/paper/2105.00894/) (2021)
- [MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation](https://mlsanity.com/paper/2203.16912/) (2022)
- [Domain Knowledge Injection in Bayesian Search for New Materials](https://mlsanity.com/paper/2311.15162/) (2023)
- [Data-driven Prior Learning for Bayesian Optimisation](https://mlsanity.com/paper/2311.14653/) (2023)

## Abstract (arXiv)

Bayesian optimisation is a popular approach for optimising expensive black-box functions. The next location to be evaluated is selected via maximising an acquisition function that balances exploitation and exploration. Gaussian processes, the surrogate models of choice in Bayesian optimisation, are often used with a constant prior mean function equal to the arithmetic mean of the observed function values. We show that the rate of convergence can depend sensitively on the choice of mean function. We empirically investigate 8 mean functions (constant functions equal to the arithmetic mean, minimum, median and maximum of the observed function evaluations, linear, quadratic polynomials, random forests and RBF networks), using 10 synthetic test problems and two real-world problems, and using the Expected Improvement and Upper Confidence Bound acquisition functions. We find that for design dimensions ≥5 using a constant mean function equal to the worst observed quality value is consistently the best choice on the synthetic problems considered. We argue that this worst-observed-quality function promotes exploitation leading to more rapid convergence. However, for the real-world tasks the more complex mean functions capable of modelling the fitness landscape may be effective, although there is no clearly optimum choice.