# Approximation of differential entropy in Bayesian optimal experimental design

arXiv: https://arxiv.org/abs/2510.00734 · HTML: https://mlsanity.com/paper/2510.00734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 60 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 14 of 14)

- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Variational Bayesian Optimal Experimental Design](https://mlsanity.com/paper/1903.05480/) (2019)
- [A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments](https://mlsanity.com/paper/1911.00294/) (2019)
- [Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation](https://mlsanity.com/paper/2002.08129/) (2020)
- [Optimal experimental design: Formulations and computations](https://mlsanity.com/paper/2407.16212/) (2024)
- [Estimating Mixture Entropy with Pairwise Distances](https://mlsanity.com/paper/1706.02419/) (2017)
- [Unbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs](https://mlsanity.com/paper/2005.08414/) (2020)
- [Large-scale Bayesian optimal experimental design with derivative-informed projected neural network](https://mlsanity.com/paper/2201.07925/) (2022)
- [Multilevel Monte Carlo estimation of expected information gains](https://mlsanity.com/paper/1811.07546/) (2018)
- [Edge-promoting adaptive Bayesian experimental design for X-ray imaging](https://mlsanity.com/paper/2104.00301/) (2021)
- [Probabilistic Bayesian optimal experimental design using conditional normalizing flows](https://mlsanity.com/paper/2402.18337/) (2024)
- [Sequential infinite-dimensional Bayesian optimal experimental design with derivative-informed latent attention neural operator](https://mlsanity.com/paper/2409.09141/) (2024)
- [Neural Joint Entropy Estimation](https://mlsanity.com/paper/2012.11197/) (2020)
- [Bayesian optimal experimental design with Wasserstein information criteria](https://mlsanity.com/paper/2504.10092/) (2025)

## Abstract (arXiv)

Bayesian optimal experimental design provides a principled framework for selecting experimental configurations that maximize obtained information. In this work, we consider settings where the likelihood entropy is independent of the design or can be evaluated explicitly, which applies, e.g. to additive noise models with design-independent or explicitly computable noise entropy. This reduces the problem to maximum entropy estimation, where the nested integral defining the expected information gain collapses into a single integral. Our study is motivated by large-scale inference problems, such as inverse problems, where the computational cost is dominated by expensive likelihood evaluations. Under additive Gaussian noise model, we propose a computational approach in which the evidence density is approximated by a Monte Carlo or quasi-Monte Carlo surrogate, while the differential entropy is evaluated using standard methods without additional likelihood evaluations. We prove that this strategy achieves convergence rates that are comparable to, or better than, state-of-the-art methods for full expected information gain estimation, particularly when the cost of entropy evaluation is negligible. Our approach relies only on mild smoothness of the forward map and avoids stronger technical assumptions required in earlier work. We also present numerical experiments, which confirm some of our theoretical findings.