# Rollout Algorithms and Approximate Dynamic Programming for Bayesian Optimization and Sequential Estimation

arXiv: https://arxiv.org/abs/2212.07998 · HTML: https://mlsanity.com/paper/2212.07998/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 15 Dec 2022.

## Builds on (most cited first, 11 of 11)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees](https://mlsanity.com/paper/2006.15779/) (2020)
- [BINOCULARS for Efficient, Nonmyopic Sequential Experimental Design](https://mlsanity.com/paper/1909.04568/) (2019)
- [Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control](https://mlsanity.com/paper/2108.10315/) (2021)
- [Efficient Rollout Strategies for Bayesian Optimization](https://mlsanity.com/paper/2002.10539/) (2020)
- [A Nonmyopic Approach to Cost-Constrained Bayesian Optimization](https://mlsanity.com/paper/2106.06079/) (2021)
- [Multiagent Rollout and Policy Iteration for POMDP with Application to Multi-Robot Repair Problems](https://mlsanity.com/paper/2011.04222/) (2020)
- [Multiagent Value Iteration Algorithms in Dynamic Programming and Reinforcement Learning](https://mlsanity.com/paper/2005.01627/) (2020)
- [Reinforcement Learning Methods for Wordle: A POMDP/Adaptive Control Approach](https://mlsanity.com/paper/2211.10298/) (2022)
- [Optimizing Consensus-based Multi-target Tracking with Multiagent Rollout Control Policies](https://mlsanity.com/paper/2102.02919/) (2021)
- [SMA-NBO: A Sequential Multi-Agent Planning with Nominal Belief-State Optimization in Target Tracking](https://mlsanity.com/paper/2203.01507/) (2022)

## Built on by (most cited first, 3 of 3)

- [Optimal and Efficient Binary Questioning for Human-in-the-Loop Annotation](https://mlsanity.com/paper/2307.01578/) (2023)
- [Nonmyopic Global Optimisation via Approximate Dynamic Programming](https://mlsanity.com/paper/2412.04882/) (2024)
- [Online Identification of IT Systems through Active Causal Learning](https://mlsanity.com/paper/2509.02130/) (2025)

## Abstract (arXiv)

We provide a unifying approximate dynamic programming framework that applies to a broad variety of problems involving sequential estimation. We consider first the construction of surrogate cost functions for the purposes of optimization, and we focus on the special case of Bayesian optimization, using the rollout algorithm and some of its variations. We then discuss the more general case of sequential estimation of a random vector using optimal measurement selection, and its application to problems of stochastic and adaptive control. We distinguish between adaptive control of deterministic and stochastic systems: the former are better suited for the use of rollout, while the latter are well suited for the use of rollout with certainty equivalence approximations. As an example of the deterministic case, we discuss sequential decoding problems, and a rollout algorithm for the approximate solution of the Wordle and Mastermind puzzles, recently developed in the paper [BBB22].