# Why Non-myopic Bayesian Optimization is Promising and How Far Should We Look-ahead? A Study via Rollout

arXiv: https://arxiv.org/abs/1911.01004 · HTML: https://mlsanity.com/paper/1911.01004/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2019.

## Built on by (most cited first, 23 of 23)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization](https://mlsanity.com/paper/2106.04335/) (2021)
- [Multi-Step Budgeted Bayesian Optimization with Unknown Evaluation Costs](https://mlsanity.com/paper/2111.06537/) (2021)
- [Cost-aware Bayesian Optimization via the Pandora's Box Gittins Index](https://mlsanity.com/paper/2406.20062/) (2024)
- [SnAKe: Bayesian Optimization with Pathwise Exploration](https://mlsanity.com/paper/2202.00060/) (2022)
- [Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design](https://mlsanity.com/paper/2306.14348/) (2023)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [Bayesian Optimization Over Iterative Learners with Structured Responses: A Budget-aware Planning Approach](https://mlsanity.com/paper/2206.12708/) (2022)
- [MONGOOSE: Path-wise Smooth Bayesian Optimisation via Meta-learning](https://mlsanity.com/paper/2302.11533/) (2023)
- [Statistically Efficient Bayesian Sequential Experiment Design via Reinforcement Learning with Cross-Entropy Estimators](https://mlsanity.com/paper/2305.18435/) (2023)
- [HyperJump: Accelerating HyperBand via Risk Modelling](https://mlsanity.com/paper/2108.02479/) (2021)
- [Towards Futuristic Autonomous Experimentation--A Surprise-Reacting Sequential Experiment Policy](https://mlsanity.com/paper/2112.00600/) (2021)
- [Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need](https://mlsanity.com/paper/2402.02111/) (2024)
- [Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits](https://mlsanity.com/paper/2111.10461/) (2021)
- [A Guide to Bayesian Optimization in Bioprocess Engineering](https://mlsanity.com/paper/2508.10642/) (2025)
- [Adaptive Sampling using POMDPs with Domain-Specific Considerations](https://mlsanity.com/paper/2109.11595/) (2021)
- [Optimal Observation-Intervention Trade-Off in Optimisation Problems with Causal Structure](https://mlsanity.com/paper/2309.02287/) (2023)
- [Deterministic Langevin Unconstrained Optimization with Normalizing Flows](https://mlsanity.com/paper/2310.00745/) (2023)
- [Information-Theoretic Safe Bayesian Optimization](https://mlsanity.com/paper/2402.15347/) (2024)
- [The Traveling Bandit: A Framework for Bayesian Optimization with Movement Costs](https://mlsanity.com/paper/2410.14533/) (2024)
- [Nonmyopic Global Optimisation via Approximate Dynamic Programming](https://mlsanity.com/paper/2412.04882/) (2024)
- [Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents](https://mlsanity.com/paper/2507.13491/) (2025)
- [Neural Nonmyopic Bayesian Optimization in Dynamic Cost Settings](https://mlsanity.com/paper/2601.06505/) (2026)

## Abstract (arXiv)

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic program (DP) that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-specified model. In this work we focus on the rollout approximation for solving the intractable DP. We first prove the improving nature of rollout in tackling lookahead BO and provide a sufficient condition for the used heuristic to be rollout improving. We then provide both a theoretical and practical guideline to decide on the rolling horizon stagewise. This guideline is built on quantifying the negative effect of a mis-specified model. To illustrate our idea, we provide case studies on both single and multi-information source BO. Empirical results show the advantageous properties of our method over several myopic and non-myopic BO algorithms.