mlsanityData to 14 Oct 2026

Variational Sequential Optimal Experimental Design using Reinforcement Learning

arXiv 2306.10430 ↗First posted 17 Jun 2023stat.ML cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

We present variational sequential optimal experimental design (vsOED), a novel method for optimally designing a finite sequence of experiments within a Bayesian framework with information-theoretic criteria. vsOED employs a one-point reward formulation with variational posterior approximations, providing a provable lower bound to the expected information gain. Numerical methods are developed following an actor-critic reinforcement learning approach, including derivation and estimation of variational and policy gradients to optimize the design policy, and posterior approximation using Gaussian mixture models and normalizing flows. vsOED accommodates nuisance parameters, implicit likelihoods, and multiple candidate models, while supporting flexible design criteria that can target designs for model discrimination, parameter inference, goal-oriented prediction, and their weighted combinations. We demonstrate vsOED across various engineering and science applications, illustrating its superior sample efficiency compared to existing sequential experimental design algorithms.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 35 of its 89 references that are in the map. First posted on arXiv 17 Jun 2023.

Builds on

35 of 89 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2015Variational Inference with Normalizing Flowscited by 2,170
  8. 2016Density estimation using Real NVPcited by 2,042
  9. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  10. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  11. 2018MINE: Mutual Information Neural Estimationcited by 683
  12. 2019On Variational Bounds of Mutual Informationcited by 559
23 more
  1. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  2. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 242
  3. 2020BayesFlow: Learning complex stochastic models with invertible neural networkscited by 144
  4. 2023Modern Bayesian Experimental Designcited by 125
  5. 2019Variational Bayesian Optimal Experimental Designcited by 116
  6. 2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 106
  7. 2022Amortized Inference for Causal Structure Learningcited by 90
  8. 2023Prediction-Oriented Bayesian Active Learningcited by 62
  9. 2021Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoodscited by 61
  10. 2019A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experimentscited by 58
  11. 2020Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimationcited by 56
  12. 2022Optimizing Sequential Experimental Design with Deep Reinforcement Learningcited by 46
  13. 2020Solving inverse problems using conditional invertible neural networkscited by 37
  14. 2019HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inferencecited by 31
  15. 2016Sequential Bayesian optimal experimental design via approximate dynamic programmingcited by 28
  16. 2021Gradient-based Bayesian Experimental Design for Implicit Models using Mutual Information Lower Boundscited by 27
  17. 2021Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learningcited by 24
  18. 2020Sequential Bayesian Experimental Design for Implicit Models via Mutual Informationcited by 23
  19. 2018Goal-Oriented Optimal Design of Experiments for Large-Scale Bayesian Linear Inverse Problemscited by 19
  20. 2019A layered multiple importance sampling scheme for focused optimal Bayesian experimental designcited by 9
  21. 2019Generalizing Information to the Evolution of Rational Beliefcited by 6
  22. 2021An efficient method for goal-oriented linear Bayesian optimal experimental design: Application to optimal sensor placemencited by 5
  23. 2022A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2025ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisitioncited by 8
  2. 2025JADAI: Jointly Amortizing Adaptive Design and Bayesian Inferencecited by 4
  3. 2024A Likelihood-Free Approach to Goal-Oriented Bayesian Optimal Experimental Designcited by 2
  4. 2025Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Designcited by 2
  5. 2025Goal-Oriented Sequential Bayesian Experimental Design for Causal Learningcited by 2
  6. 2026Constrained Bayesian Experimental Design via Online Planningcited by 2
  7. 2026Efficient Adaptive Data Acquisition via Pretrained Belief Representationscited by 2
  8. 2026Sequential Bayesian Optimal Experimental Design in Infinite Dimensions via Policy Gradient Reinforcement Learningcited by 1