# Efficient Rollout Strategies for Bayesian Optimization

arXiv: https://arxiv.org/abs/2002.10539 · HTML: https://mlsanity.com/paper/2002.10539/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2020.

## Built on by (most cited first, 19 of 19)

- [A Nonmyopic Approach to Cost-Constrained Bayesian Optimization](https://mlsanity.com/paper/2106.06079/) (2021)
- [Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs](https://mlsanity.com/paper/2111.06537/) (2021)
- [Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design](https://mlsanity.com/paper/2306.14348/) (2023)
- [Bayesian Optimization Over Iterative Learners with Structured Responses: A Budget-aware Planning Approach](https://mlsanity.com/paper/2206.12708/) (2022)
- [MONGOOSE: Path-wise Smooth Bayesian Optimisation via Meta-learning](https://mlsanity.com/paper/2302.11533/) (2023)
- [EARL-BO: Reinforcement Learning for Multi-Step Lookahead, High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2411.00171/) (2024)
- [The Gittins Index: A Design Principle for Decision-Making Under Uncertainty](https://mlsanity.com/paper/2506.10872/) (2025)
- [Towards Futuristic Autonomous Experimentation--A Surprise-Reacting Sequential Experiment Policy](https://mlsanity.com/paper/2112.00600/) (2021)
- [Two-step Lookahead Bayesian Optimization with Inequality Constraints](https://mlsanity.com/paper/2112.02833/) (2021)
- [Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need](https://mlsanity.com/paper/2402.02111/) (2024)
- [B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search](https://mlsanity.com/paper/2202.03005/) (2022)
- [Rollout Algorithms and Approximate Dynamic Programming for Bayesian Optimization and Sequential Estimation](https://mlsanity.com/paper/2212.07998/) (2022)
- [Scalable Bayesian optimization with high-dimensional outputs using randomized prior networks](https://mlsanity.com/paper/2302.07260/) (2023)
- [Constrained Bayesian Experimental Design via Online Planning](https://mlsanity.com/paper/2605.26990/) (2026)
- [Automatic model training under restrictive time constraints](https://mlsanity.com/paper/2104.10746/) (2021)
- [Differentiating Policies for Non-Myopic Bayesian Optimization](https://mlsanity.com/paper/2408.07812/) (2024)
- [Nonmyopic Global Optimisation via Approximate Dynamic Programming](https://mlsanity.com/paper/2412.04882/) (2024)
- [Non-Myopic Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2412.08085/) (2024)
- [Neural Nonmyopic Bayesian Optimization in Dynamic Cost Settings](https://mlsanity.com/paper/2601.06505/) (2026)

## Abstract (arXiv)

Bayesian optimization (BO) is a class of sample-efficient global optimization methods, where a probabilistic model conditioned on previous observations is used to determine future evaluations via the optimization of an acquisition function. Most acquisition functions are myopic, meaning that they only consider the impact of the next function evaluation. Non-myopic acquisition functions consider the impact of the next h function evaluations and are typically computed through rollout, in which h steps of BO are simulated. These rollout acquisition functions are defined as h-dimensional integrals, and are expensive to compute and optimize. We show that a combination of quasi-Monte Carlo, common random numbers, and control variates significantly reduce the computational burden of rollout. We then formulate a policy-search based approach that removes the need to optimize the rollout acquisition function. Finally, we discuss the qualitative behavior of rollout policies in the setting of multi-modal objectives and model error.