# Sequential Bayesian Optimal Experimental Design in Infinite Dimensions via Policy Gradient Reinforcement Learning

arXiv: https://arxiv.org/abs/2601.05868 · HTML: https://mlsanity.com/paper/2601.05868/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 31 of its 57 references that are in the map.
- First posted on arXiv 9 Jan 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [Scalable Transformer for PDE Surrogate Modeling](https://mlsanity.com/paper/2305.17560/) (2023)
- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Variational Bayesian Optimal Experimental Design](https://mlsanity.com/paper/1903.05480/) (2019)
- [Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design](https://mlsanity.com/paper/2103.02438/) (2021)
- [Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods](https://mlsanity.com/paper/2111.02329/) (2021)
- [Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation](https://mlsanity.com/paper/2002.08129/) (2020)
- [Optimizing Sequential Experimental Design with Deep Reinforcement Learning](https://mlsanity.com/paper/2202.00821/) (2022)
- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Sequential Bayesian optimal experimental design via approximate dynamic programming](https://mlsanity.com/paper/1604.08320/) (2016)
- [Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learning](https://mlsanity.com/paper/2110.15335/) (2021)
- [Goal-Oriented Optimal Design of Experiments for Large-Scale Bayesian Linear Inverse Problems](https://mlsanity.com/paper/1802.06517/) (2018)
- [Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators](https://mlsanity.com/paper/2305.20053/) (2023)
- [Derivative-enhanced Deep Operator Network](https://mlsanity.com/paper/2402.19242/) (2024)
- [Large-scale Bayesian optimal experimental design with derivative-informed projected neural network](https://mlsanity.com/paper/2201.07925/) (2022)
- [Learning High-Dimensional Parametric Maps via Reduced Basis Adaptive Residual Networks](https://mlsanity.com/paper/2112.07096/) (2021)
- [A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-dimensional Bayesian Nonlinear Inverse Problems](https://mlsanity.com/paper/1410.5899/) (2014)
- [Robust Expected Information Gain for Optimal Bayesian Experimental Design Using Ambiguity Sets](https://mlsanity.com/paper/2205.09914/) (2022)
- [Edge-promoting adaptive Bayesian experimental design for X-ray imaging](https://mlsanity.com/paper/2104.00301/) (2021)
- [Accurate, scalable, and efficient Bayesian optimal experimental design with derivative-informed neural operators](https://mlsanity.com/paper/2312.14810/) (2023)
- [Probabilistic Bayesian optimal experimental design using conditional normalizing flows](https://mlsanity.com/paper/2402.18337/) (2024)
- [Sequential infinite-dimensional Bayesian optimal experimental design with derivative-informed latent attention neural operator](https://mlsanity.com/paper/2409.09141/) (2024)
- [A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized ℓ0-sparsification](https://mlsanity.com/paper/1308.4084/) (2013)
- [Variational Sequential Optimal Experimental Design using Reinforcement Learning](https://mlsanity.com/paper/2306.10430/) (2023)
- [Mitigating the Influence of the Boundary on PDE-based Covariance Operators](https://mlsanity.com/paper/1610.05280/) (2016)
- [Batch greedy maximization of non-submodular functions: Guarantees and applications to experimental design](https://mlsanity.com/paper/2006.04554/) (2020)

## Built on by (most cited first, 1 of 1)

- [Toward AI-Driven Digital Twins for Metropolitan Floods: A Conditional Latent Dynamics Network Surrogate of the Shallow Water Equations](https://mlsanity.com/paper/2605.13761/) (2026)

## Abstract (arXiv)

Sequential Bayesian optimal experimental design (SBOED) for PDE-governed inverse problems is computationally challenging, especially for infinite-dimensional random field parameters. High-fidelity approaches require repeated forward and adjoint PDE solves inside nested Bayesian inversion and design loops. We formulate SBOED as a finite-horizon Markov decision process and learn an amortized design policy via policy-gradient reinforcement learning (PGRL), enabling online design selection from the experiment history without repeatedly solving an SBOED optimization problem. To make policy training and reward evaluation scalable, we combine dual dimension reduction -- active subspace projection for the parameter and principal component analysis for the state -- with an adjusted derivative-informed latent attention neural operator (LANO) surrogate that predicts both the parameter-to-solution map and its Jacobian. We use a Laplace-based D-optimality reward while noting that, in general, other expected-information-gain utilities such as KL divergence can also be used within the same framework. We further introduce an eigenvalue-based evaluation strategy that uses prior samples as proxies for maximum a posteriori (MAP) points, avoiding repeated MAP solves while retaining accurate information-gain estimates. Numerical experiments on sequential multi-sensor placement for contaminant source tracking demonstrate approximately 100× speedup over high-fidelity finite element methods, improved performance over random sensor placements, and physically interpretable policies that discover an ``upstream'' tracking strategy.