# Simulation-based Inference with the Generalized Kullback-Leibler Divergence

arXiv: https://arxiv.org/abs/2310.01808 · HTML: https://mlsanity.com/paper/2310.01808/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 70 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 30 of 34)

- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Automatic Posterior Transformation for Likelihood-Free Inference](https://mlsanity.com/paper/1905.07488/) (2019)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Flexible statistical inference for mechanistic models of neural dynamics](https://mlsanity.com/paper/1711.01861/) (2017)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [Telescoping Density-Ratio Estimation](https://mlsanity.com/paper/2006.12204/) (2020)
- [Likelihood-free inference by ratio estimation](https://mlsanity.com/paper/1611.10242/) (2016)
- [Real-time gravitational-wave science with neural posterior estimation](https://mlsanity.com/paper/2106.12594/) (2021)
- [Generalized Energy Based Models](https://mlsanity.com/paper/2003.05033/) (2020)
- [Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models](https://mlsanity.com/paper/2210.04872/) (2022)
- [Sampling-Based Accuracy Testing of Posterior Estimators for General Inference](https://mlsanity.com/paper/2302.03026/) (2023)
- [Variational methods for simulation-based inference](https://mlsanity.com/paper/2203.04176/) (2022)
- [GATSBI: Generative Adversarial Training for Simulation-Based Inference](https://mlsanity.com/paper/2203.06481/) (2022)
- [The Expressive Power of a Class of Normalizing Flow Models](https://mlsanity.com/paper/2006.00392/) (2020)
- [Truncated Marginal Neural Ratio Estimation](https://mlsanity.com/paper/2107.01214/) (2021)
- [Diagnostics for Conditional Density Models and Bayesian Inference Algorithms](https://mlsanity.com/paper/2102.10473/) (2021)
- [Score Matched Neural Exponential Families for Likelihood-Free Inference](https://mlsanity.com/paper/2012.10903/) (2020)
- [Fast and Credible Likelihood-Free Cosmology with Truncated Marginal Neural Ratio Estimation](https://mlsanity.com/paper/2111.08030/) (2021)
- [E-Valuating Classifier Two-Sample Tests](https://mlsanity.com/paper/2210.13027/) (2022)
- [Balancing Simulation-based Inference for Conservative Posteriors](https://mlsanity.com/paper/2304.10978/) (2023)

## Built on by (most cited first, 3 of 3)

- [SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data](https://mlsanity.com/paper/2505.20347/) (2025)
- [Alignist: CAD-Informed Orientation Distribution Estimation by Fusing Shape and Correspondences](https://mlsanity.com/paper/2409.06683/) (2024)
- [Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design](https://mlsanity.com/paper/2502.08004/) (2025)

## Abstract (arXiv)

In Simulation-based Inference, the goal is to solve the inverse problem when the likelihood is only known implicitly. Neural Posterior Estimation commonly fits a normalized density estimator as a surrogate model for the posterior. This formulation cannot easily fit unnormalized surrogates because it optimizes the Kullback-Leibler divergence. We propose to optimize a generalized Kullback-Leibler divergence that accounts for the normalization constant in unnormalized distributions. The objective recovers Neural Posterior Estimation when the model class is normalized and unifies it with Neural Ratio Estimation, combining both into a single objective. We investigate a hybrid model that offers the best of both worlds by learning a normalized base distribution and a learned ratio. We also present benchmark results.