# Gradient-based Bayesian Experimental Design for Implicit Models using Mutual Information Lower Bounds

arXiv: https://arxiv.org/abs/2105.04379 · HTML: https://mlsanity.com/paper/2105.04379/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 26 of 26)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information](https://mlsanity.com/paper/2006.12013/) (2020)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Flexible statistical inference for mechanistic models of neural dynamics](https://mlsanity.com/paper/1711.01861/) (2017)
- [Understanding the Limitations of Variational Mutual Information Estimators](https://mlsanity.com/paper/1910.06222/) (2019)
- [Variational Bayesian Optimal Experimental Design](https://mlsanity.com/paper/1903.05480/) (2019)
- [Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design](https://mlsanity.com/paper/2103.02438/) (2021)
- [Likelihood-free inference by ratio estimation](https://mlsanity.com/paper/1611.10242/) (2016)
- [Neural Approximate Sufficient Statistics for Implicit Models](https://mlsanity.com/paper/2010.10079/) (2020)
- [Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models](https://mlsanity.com/paper/1501.03291/) (2015)
- [Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation](https://mlsanity.com/paper/2002.08129/) (2020)
- [Efficient Bayesian Experimental Design for Implicit Models](https://mlsanity.com/paper/1810.09912/) (2018)
- [Sequential Bayesian Experimental Design for Implicit Models via Mutual Information](https://mlsanity.com/paper/2003.09379/) (2020)
- [Bayesian design of experiments for intractable likelihood models using coupled auxiliary models and multivariate emulation](https://mlsanity.com/paper/1803.07018/) (2018)
- [Adaptive Gaussian Copula ABC](https://mlsanity.com/paper/1902.10704/) (2019)
- [A Scalable Gradient-Free Method for Bayesian Experimental Design with Implicit Models](https://mlsanity.com/paper/2103.08026/) (2021)
- [Gaussian process modeling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria](https://mlsanity.com/paper/1610.06462/) (2016)
- [Optimal Bayesian design for model discrimination via classification](https://mlsanity.com/paper/1809.05301/) (2018)

## Built on by (most cited first, 27 of 27)

- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods](https://mlsanity.com/paper/2111.02329/) (2021)
- [Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization](https://mlsanity.com/paper/2107.01131/) (2021)
- [Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models](https://mlsanity.com/paper/2203.04272/) (2022)
- [Differentiable Multi-Target Causal Bayesian Experimental Design](https://mlsanity.com/paper/2302.10607/) (2023)
- [Step-DAD: Semi-Amortized Policy-Based Bayesian Experimental Design](https://mlsanity.com/paper/2507.14057/) (2025)
- [Bayesian Experimental Design via Contrastive Diffusions](https://mlsanity.com/paper/2410.11826/) (2024)
- [Variational Sequential Optimal Experimental Design using Reinforcement Learning](https://mlsanity.com/paper/2306.10430/) (2023)
- [Statistical applications of contrastive learning](https://mlsanity.com/paper/2204.13999/) (2022)
- [On Estimating the Gradient of the Expected Information Gain in Bayesian Experimental Design](https://mlsanity.com/paper/2308.09888/) (2023)
- [PASOA- PArticle baSed Bayesian Optimal Adaptive design](https://mlsanity.com/paper/2402.07160/) (2024)
- [CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Design](https://mlsanity.com/paper/2302.14015/) (2023)
- [Online simulator-based experimental design for cognitive model selection](https://mlsanity.com/paper/2303.02227/) (2023)
- [Designing Optimal Behavioral Experiments Using Machine Learning](https://mlsanity.com/paper/2305.07721/) (2023)
- [Performance Comparisons of Reinforcement Learning Algorithms for Sequential Experimental Design](https://mlsanity.com/paper/2503.05905/) (2025)
- [A Geometric Approach to Optimal Experimental Design](https://mlsanity.com/paper/2510.14848/) (2025)
- [Nonlinear Bayesian optimal experimental design using logarithmic Sobolev inequalities](https://mlsanity.com/paper/2402.15053/) (2024)
- [A Likelihood-Free Approach to Goal-Oriented Bayesian Optimal Experimental Design](https://mlsanity.com/paper/2408.09582/) (2024)
- [Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design](https://mlsanity.com/paper/2502.08004/) (2025)
- [Action-BED: Task-Driven Bayesian Experimental Design with Singly Intractable Objectives](https://mlsanity.com/paper/2606.23662/) (2026)
- [MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories](https://mlsanity.com/paper/2106.01808/) (2021)
- [Stochastic Gradient Bayesian Optimal Experimental Designs for Simulation-based Inference](https://mlsanity.com/paper/2306.15731/) (2023)
- [Amortised Experimental Design and Parameter Estimation for User Models of Pointing](https://mlsanity.com/paper/2307.09878/) (2023)
- [Online Bayesian Experimental Design for Partially Observed Dynamical Systems](https://mlsanity.com/paper/2511.04403/) (2025)
- [An Optimal Likelihood Free Method for Biological Model Selection](https://mlsanity.com/paper/2208.02344/) (2022)
- [Case-Guided Sequential Assay Planning in Drug Discovery](https://mlsanity.com/paper/2601.14710/) (2026)
- [Goal-driven Bayesian Optimal Experimental Design for Robust Decision-Making Under Model Uncertainty](https://mlsanity.com/paper/2605.26093/) (2026)

## Abstract (arXiv)

We introduce a framework for Bayesian experimental design (BED) with implicit models, where the data-generating distribution is intractable but sampling from it is still possible. In order to find optimal experimental designs for such models, our approach maximises mutual information lower bounds that are parametrised by neural networks. By training a neural network on sampled data, we simultaneously update network parameters and designs using stochastic gradient-ascent. The framework enables experimental design with a variety of prominent lower bounds and can be applied to a wide range of scientific tasks, such as parameter estimation, model discrimination and improving future predictions. Using a set of intractable toy models, we provide a comprehensive empirical comparison of prominent lower bounds applied to the aforementioned tasks. We further validate our framework on a challenging system of stochastic differential equations from epidemiology.