# Neural Empirical Bayes: Source Distribution Estimation and its Applications to Simulation-Based Inference

arXiv: https://arxiv.org/abs/2011.05836 · HTML: https://mlsanity.com/paper/2011.05836/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2020.

## Built on by (most cited first, 21 of 21)

- [A Living Review of Machine Learning for Particle Physics](https://mlsanity.com/paper/2102.02770/) (2021)
- [Learning Diffusion Priors from Observations by Expectation Maximization](https://mlsanity.com/paper/2405.13712/) (2024)
- [The Landscape of Unfolding with Machine Learning](https://mlsanity.com/paper/2404.18807/) (2024)
- [New directions for surrogate models and differentiable programming for High Energy Physics detector simulation](https://mlsanity.com/paper/2203.08806/) (2022)
- [Simulation-based Inference for Cardiovascular Models](https://mlsanity.com/paper/2307.13918/) (2023)
- [Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation](https://mlsanity.com/paper/2402.07808/) (2024)
- [Unifying Simulation and Inference with Normalizing Flows](https://mlsanity.com/paper/2404.18992/) (2024)
- [Generative Unfolding with Distribution Mapping](https://mlsanity.com/paper/2411.02495/) (2024)
- [Scaling laws for amplitude surrogates](https://mlsanity.com/paper/2601.13308/) (2026)
- [Unbinned Profiled Unfolding](https://mlsanity.com/paper/2302.05390/) (2023)
- [Leveraging Cardiovascular Simulations for In-Vivo Prediction of Cardiac Biomarkers](https://mlsanity.com/paper/2412.17542/) (2024)
- [Simulation-Prior Independent Neural Unfolding Procedure](https://mlsanity.com/paper/2507.15084/) (2025)
- [Normalizing Flows for Hierarchical Bayesian Analysis: A Gravitational Wave Population Study](https://mlsanity.com/paper/2211.09008/) (2022)
- [Predictive variational inference: Learn the predictively optimal posterior distribution](https://mlsanity.com/paper/2410.14843/) (2024)
- [Discriminative calibration: Check Bayesian computation from simulations and flexible classifier](https://mlsanity.com/paper/2305.14593/) (2023)
- [Universal priors: solving empirical Bayes via Bayesian inference and pretraining](https://mlsanity.com/paper/2602.15136/) (2026)
- [An Introduction to Bayesian and Frequentist Simulation-Based Inference with Machine Learning](https://mlsanity.com/paper/2607.21702/) (2026)
- [Simulation-Based Empirical Bayes](https://mlsanity.com/paper/2607.21843/) (2026)
- [Designing Observables for Measurements with Deep Learning](https://mlsanity.com/paper/2310.08717/) (2023)
- [Neural-g: A Deep Learning Framework for Mixing Density Estimation](https://mlsanity.com/paper/2406.05986/) (2024)
- [Source Distribution Estimation by Posterior Averaging](https://mlsanity.com/paper/2609.02622/) (2026)

## Abstract (arXiv)

We revisit empirical Bayes in the absence of a tractable likelihood function, as is typical in scientific domains relying on computer simulations. We investigate how the empirical Bayesian can make use of neural density estimators first to use all noise-corrupted observations to estimate a prior or source distribution over uncorrupted samples, and then to perform single-observation posterior inference using the fitted source distribution. We propose an approach based on the direct maximization of the log-marginal likelihood of the observations, examining both biased and de-biased estimators, and comparing to variational approaches. We find that, up to symmetries, a neural empirical Bayes approach recovers ground truth source distributions. With the learned source distribution in hand, we show the applicability to likelihood-free inference and examine the quality of the resulting posterior estimates. Finally, we demonstrate the applicability of Neural Empirical Bayes on an inverse problem from collider physics.