# Reinforcement Learning based Sequential Batch-sampling for Bayesian Optimal Experimental Design

arXiv: https://arxiv.org/abs/2112.10944 · HTML: https://mlsanity.com/paper/2112.10944/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 53 references that are in the map.
- First posted on arXiv 21 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [Batch Bayesian Optimization via Local Penalization](https://mlsanity.com/paper/1505.08052/) (2015)
- [Learning to Optimize Neural Nets](https://mlsanity.com/paper/1703.00441/) (2017)
- [Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learning](https://mlsanity.com/paper/2110.15335/) (2021)
- [Stress field prediction in fiber-reinforced composite materials using a deep learning approach](https://mlsanity.com/paper/2111.05271/) (2021)
- [Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints](https://mlsanity.com/paper/2007.01380/) (2020)
- [Inverse Aerodynamic Design of Gas Turbine Blades using Probabilistic Machine Learning](https://mlsanity.com/paper/2108.10163/) (2021)
- [An efficient optimization based microstructure reconstruction approach with multiple loss functions](https://mlsanity.com/paper/2102.02407/) (2021)
- [Efficient Bayesian experimentation using an expected information gain lower bound](https://mlsanity.com/paper/1506.00053/) (2015)
- [Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes](https://mlsanity.com/paper/2008.02386/) (2020)

## Built on by (most cited first, 1 of 1)

- [Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models](https://mlsanity.com/paper/2203.04272/) (2022)

## Abstract (arXiv)

Engineering problems that are modeled using sophisticated mathematical methods or are characterized by expensive-to-conduct tests or experiments, are encumbered with limited budget or finite computational resources. Moreover, practical scenarios in the industry, impose restrictions, based on logistics and preference, on the manner in which the experiments can be conducted. For example, material supply may enable only a handful of experiments in a single-shot or in the case of computational models one may face significant wait-time based on shared computational resources. In such scenarios, one usually resorts to performing experiments in a manner that allows for maximizing one's state-of-knowledge while satisfying the above mentioned practical constraints. Sequential design of experiments (SDOE) is a popular suite of methods, that has yielded promising results in recent years across different engineering and practical problems. A common strategy, that leverages Bayesian formalism is the Bayesian SDOE, which usually works best in the one-step-ahead or myopic scenario of selecting a single experiment at each step of a sequence of experiments. In this work, we aim to extend the SDOE strategy, to query the experiment or computer code at a batch of inputs. To this end, we leverage deep reinforcement learning (RL) based policy gradient methods, to propose batches of queries that are selected taking into account entire budget in hand. The algorithm retains the sequential nature, inherent in the SDOE, while incorporating elements of reward based on task from the domain of deep RL. A unique capability of the proposed methodology is its ability to be applied to multiple tasks, for example optimization of a function, once its trained. We demonstrate the performance of the proposed algorithm on a synthetic problem, and a challenging high-dimensional engineering problem.