# Sequential Bayesian optimal experimental design via approximate dynamic programming

arXiv: https://arxiv.org/abs/1604.08320 · HTML: https://mlsanity.com/paper/1604.08320/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2016.

## Built on by (most cited first, 28 of 28)

- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design](https://mlsanity.com/paper/2103.02438/) (2021)
- [Optimizing Sequential Experimental Design with Deep Reinforcement Learning](https://mlsanity.com/paper/2202.00821/) (2022)
- [Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learning](https://mlsanity.com/paper/2110.15335/) (2021)
- [BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design](https://mlsanity.com/paper/2508.21184/) (2025)
- [Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models](https://mlsanity.com/paper/2203.04272/) (2022)
- [Nesting Particle Filters for Experimental Design in Dynamical Systems](https://mlsanity.com/paper/2402.07868/) (2024)
- [Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design](https://mlsanity.com/paper/2507.14057/) (2025)
- [Amortized Active Causal Induction with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.16718/) (2024)
- [Characterizing the robustness of Bayesian adaptive experimental designs to active learning bias](https://mlsanity.com/paper/2205.13698/) (2022)
- [Variational Sequential Optimal Experimental Design using Reinforcement Learning](https://mlsanity.com/paper/2306.10430/) (2023)
- [Design of experiments for the calibration of history-dependent models via deep reinforcement learning and an enhanced Kalman filter](https://mlsanity.com/paper/2209.13126/) (2022)
- [Active Learning of Model Discrepancy with Bayesian Experimental Design](https://mlsanity.com/paper/2502.05372/) (2025)
- [Experimentation Platforms Meet Reinforcement Learning: Bayesian Sequential Decision-Making for Continuous Monitoring](https://mlsanity.com/paper/2304.00420/) (2023)
- [Recursive Nested Filtering for Efficient Amortized Bayesian Experimental Design](https://mlsanity.com/paper/2409.05354/) (2024)
- [Performance Comparisons of Reinforcement Learning Algorithms for Sequential Experimental Design](https://mlsanity.com/paper/2503.05905/) (2025)
- [Loss-Driven Bayesian Active Learning](https://mlsanity.com/paper/2604.11995/) (2026)
- [Constrained Bayesian Experimental Design via Online Planning](https://mlsanity.com/paper/2605.26990/) (2026)
- [Efficient Adaptive Data Acquisition via Pretrained Belief Representations](https://mlsanity.com/paper/2606.25197/) (2026)
- [Sequential Bayesian Optimal Experimental Design in Infinite Dimensions via Policy Gradient Reinforcement Learning](https://mlsanity.com/paper/2601.05868/) (2026)
- [Optimal Experimental Design for Reliable Learning of History-Dependent Constitutive Laws](https://mlsanity.com/paper/2603.12365/) (2026)
- [Sequential Bayesian experimental designs via reinforcement learning](https://mlsanity.com/paper/2202.07472/) (2022)
- [Dimensionality Collapse: Optimal Measurement Selection for Low-Error Infinite-Horizon Forecasting](https://mlsanity.com/paper/2303.15407/) (2023)
- [Gradient-Free Sequential Bayesian Experimental Design via Interacting Particle Systems](https://mlsanity.com/paper/2504.13320/) (2025)
- [Data-driven Sensor Placement for Predictive Applications: A Correlation-Assisted Attribution Framework (CAAF)](https://mlsanity.com/paper/2510.22517/) (2025)
- [Robust Sequential Experimental Design for A/B Testing](https://mlsanity.com/paper/2605.12899/) (2026)
- [Edge-Aligned Beam Placement in Scanning Probe Tomography via Reconstruction-Free Sequential Design of Experiments](https://mlsanity.com/paper/2606.21713/) (2026)
- [Deep adaptive design with an evidential bias criterion](https://mlsanity.com/paper/2608.16466/) (2026)

## Abstract (arXiv)

The design of multiple experiments is commonly undertaken via suboptimal strategies, such as batch (open-loop) design that omits feedback or greedy (myopic) design that does not account for future effects. This paper introduces new strategies for the optimal design of sequential experiments. First, we rigorously formulate the general sequential optimal experimental design (sOED) problem as a dynamic program. Batch and greedy designs are shown to result from special cases of this formulation. We then focus on sOED for parameter inference, adopting a Bayesian formulation with an information theoretic design objective. To make the problem tractable, we develop new numerical approaches for nonlinear design with continuous parameter, design, and observation spaces. We approximate the optimal policy by using backward induction with regression to construct and refine value function approximations in the dynamic program. The proposed algorithm iteratively generates trajectories via exploration and exploitation to improve approximation accuracy in frequently visited regions of the state space. Numerical results are verified against analytical solutions in a linear-Gaussian setting. Advantages over batch and greedy design are then demonstrated on a nonlinear source inversion problem where we seek an optimal policy for sequential sensing.