# Variational Bayesian Optimal Experimental Design with Normalizing Flows

arXiv: https://arxiv.org/abs/2404.13056 · HTML: https://mlsanity.com/paper/2404.13056/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 69 references that are in the map.
- First posted on arXiv 8 Apr 2024.

## Builds on (most cited first, 30 of 41)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [BayesFlow: Learning complex stochastic models with invertible neural networks](https://mlsanity.com/paper/2003.06281/) (2020)
- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Variational Bayesian Optimal Experimental Design](https://mlsanity.com/paper/1903.05480/) (2019)
- [On Nesting Monte Carlo Estimators](https://mlsanity.com/paper/1709.06181/) (2017)
- [Sylvester Normalizing Flows for Variational Inference](https://mlsanity.com/paper/1803.05649/) (2018)
- [Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods](https://mlsanity.com/paper/2111.02329/) (2021)
- [A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments](https://mlsanity.com/paper/1911.00294/) (2019)
- [Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation](https://mlsanity.com/paper/2002.08129/) (2020)
- [Inference via low-dimensional couplings](https://mlsanity.com/paper/1703.06131/) (2017)
- [Improving Variational Auto-Encoders using Householder Flow](https://mlsanity.com/paper/1611.09630/) (2016)
- [On the representation and learning of monotone triangular transport maps](https://mlsanity.com/paper/2009.10303/) (2020)
- [Solving inverse problems using conditional invertible neural networks](https://mlsanity.com/paper/2007.15849/) (2020)
- [Cubic-Spline Flows](https://mlsanity.com/paper/1906.02145/) (2019)
- [HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference](https://mlsanity.com/paper/1905.10687/) (2019)

## Built on by (most cited first, 13 of 13)

- [Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Models](https://mlsanity.com/paper/2407.04958/) (2024)
- [Expected information gain estimation via density approximations: Sample allocation and dimension reduction](https://mlsanity.com/paper/2411.08390/) (2024)
- [JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference](https://mlsanity.com/paper/2512.22999/) (2025)
- [Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design](https://mlsanity.com/paper/2502.08004/) (2025)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [Representative, Informative, and De-Amplifying: Requirements for Robust Bayesian Active Learning under Model Misspecification](https://mlsanity.com/paper/2506.07805/) (2025)
- [Goal-Oriented Sequential Bayesian Experimental Design for Causal Learning](https://mlsanity.com/paper/2507.07359/) (2025)
- [Optimal Experimental Design for Reliable Learning of History-Dependent Constitutive Laws](https://mlsanity.com/paper/2603.12365/) (2026)
- [Bayesian Experimental Design via Score Matching](https://mlsanity.com/paper/2607.08335/) (2026)
- [Accelerated Bayesian Optimal Experimental Design via Conditional Density Estimation and Informative Data](https://mlsanity.com/paper/2507.15235/) (2025)
- [Neural Optimal Design of Experiment for Inverse Problems](https://mlsanity.com/paper/2512.23763/) (2025)
- [Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance](https://mlsanity.com/paper/2601.16425/) (2026)
- [Goal-driven Bayesian Optimal Experimental Design for Robust Decision-Making Under Model Uncertainty](https://mlsanity.com/paper/2605.26093/) (2026)

## Abstract (arXiv)

Bayesian optimal experimental design (OED) seeks experiments that maximize the expected information gain (EIG) in model parameters. Directly estimating the EIG using nested Monte Carlo is computationally expensive and requires an explicit likelihood. Variational OED (vOED), in contrast, estimates a lower bound of the EIG without likelihood evaluations by approximating the posterior distributions with variational forms, and then tightens the bound by optimizing its variational parameters. We introduce the use of normalizing flows (NFs) for representing variational distributions in vOED; we call this approach vOED-NFs. Specifically, we adopt NFs with a conditional invertible neural network architecture built from compositions of coupling layers, and enhanced with a summary network for data dimension reduction. We present Monte Carlo estimators to the lower bound along with gradient expressions to enable a gradient-based simultaneous optimization of the variational parameters and the design variables. The vOED-NFs algorithm is then validated in two benchmark problems, and demonstrated on a partial differential equation-governed application of cathodic electrophoretic deposition and an implicit likelihood case with stochastic modeling of aphid population. The findings suggest that a composition of 4--5 coupling layers is able to achieve lower EIG estimation bias, under a fixed budget of forward model runs, compared to previous approaches. The resulting NFs produce approximate posteriors that agree well with the true posteriors, able to capture non-Gaussian and multi-modal features effectively.