# A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful

arXiv: https://arxiv.org/abs/2110.06581 · HTML: https://mlsanity.com/paper/2110.06581/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 74 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Automatic Posterior Transformation for Likelihood-Free Inference](https://mlsanity.com/paper/1905.07488/) (2019)
- [Flexible statistical inference for mechanistic models of neural dynamics](https://mlsanity.com/paper/1711.01861/) (2017)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [On Contrastive Learning for Likelihood-free Inference](https://mlsanity.com/paper/2002.03712/) (2020)
- [Approximating Likelihood Ratios with Calibrated Discriminative Classifiers](https://mlsanity.com/paper/1506.02169/) (2015)
- [Likelihood-free inference by ratio estimation](https://mlsanity.com/paper/1611.10242/) (2016)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [Diagnostics for Conditional Density Models and Bayesian Inference Algorithms](https://mlsanity.com/paper/2102.10473/) (2021)
- [A Guide to Constraining Effective Field Theories with Machine Learning](https://mlsanity.com/paper/1805.00020/) (2018)
- [Score Matched Neural Exponential Families for Likelihood-Free Inference](https://mlsanity.com/paper/2012.10903/) (2020)
- [Generalized Posteriors in Approximate Bayesian Computation](https://mlsanity.com/paper/2011.08644/) (2020)
- [Confidence Sets and Hypothesis Testing in a Likelihood-Free Inference Setting](https://mlsanity.com/paper/2002.10399/) (2020)
- [Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning](https://mlsanity.com/paper/1909.02005/) (2019)
- [Towards constraining warm dark matter with stellar streams through neural simulation-based inference](https://mlsanity.com/paper/2011.14923/) (2020)
- [Diagnostic tools of approximate Bayesian computation using the coverage property](https://mlsanity.com/paper/1301.3166/) (2013)
- [Arbitrary Marginal Neural Ratio Estimation for Simulation-based Inference](https://mlsanity.com/paper/2110.00449/) (2021)
- [Bounding the Test Log-Likelihood of Generative Models](https://mlsanity.com/paper/1311.6184/) (2013)

## Built on by (most cited first, 28 of 28)

- [All-in-one simulation-based inference](https://mlsanity.com/paper/2404.09636/) (2024)
- [Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models](https://mlsanity.com/paper/2210.04872/) (2022)
- [JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models](https://mlsanity.com/paper/2302.09125/) (2023)
- [Adversarial robustness of amortized Bayesian inference](https://mlsanity.com/paper/2305.14984/) (2023)
- [Hierarchical Neural Simulation-Based Inference Over Event Ensembles](https://mlsanity.com/paper/2306.12584/) (2023)
- [G-Sim: Generative Simulations with Large Language Models and Gradient-Free Calibration](https://mlsanity.com/paper/2506.09272/) (2025)
- [AI for Science: An Emerging Agenda](https://mlsanity.com/paper/2303.04217/) (2023)
- [Simulation-Based Inference with Quantile Regression](https://mlsanity.com/paper/2401.02413/) (2024)
- [Conformal Calibration of Statistical Confidence Sets](https://mlsanity.com/paper/2411.19368/) (2024)
- [Discriminative calibration: Check Bayesian computation from simulations and flexible classifier](https://mlsanity.com/paper/2305.14593/) (2023)
- [From Mice to Trains: Amortized Bayesian Inference on Graph Data](https://mlsanity.com/paper/2601.02241/) (2026)
- [SoftCVI: Contrastive variational inference with self-generated soft labels](https://mlsanity.com/paper/2407.15687/) (2024)
- [Physically Interpretable Probabilistic Domain Characterization](https://mlsanity.com/paper/2411.14827/) (2024)
- [A Review of Diffusion-based Simulation-Based Inference: Foundations and Applications in Non-Ideal Data Scenarios](https://mlsanity.com/paper/2512.23748/) (2025)
- [Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems](https://mlsanity.com/paper/2605.22891/) (2026)
- [An efficient likelihood-free Bayesian inference method based on sequential neural posterior estimation](https://mlsanity.com/paper/2311.12530/) (2023)
- [Leveraging Nested MLMC for Sequential Neural Posterior Estimation with Intractable Likelihoods](https://mlsanity.com/paper/2401.16776/) (2024)
- [Cost Estimation in Unit Commitment Problems Using Simulation-Based Inference](https://mlsanity.com/paper/2409.03588/) (2024)
- [Fast reconstruction of degenerate populations of conductance-based neuron models from spike times](https://mlsanity.com/paper/2509.12783/) (2025)
- [Dark Energy Survey Year 3 results: Simulation-based wCDM inference from weak lensing and galaxy clustering maps with deep learning: Analysis design](https://mlsanity.com/paper/2511.04681/) (2025)
- [Fast and Robust Simulation-Based Inference With Optimization Monte Carlo](https://mlsanity.com/paper/2511.13394/) (2025)
- [Simulation-Based Inference via Regression Projection and Batched Discrepancies](https://mlsanity.com/paper/2602.03613/) (2026)
- [Confidence Freeze: Early Success Induces a Metastable Decoupling of Metacognition and Behaviour](https://mlsanity.com/paper/2603.21043/) (2026)
- [Conservative neural posterior estimation via distributionally robust training](https://mlsanity.com/paper/2605.28516/) (2026)
- [Amortized Inference for Sampling Distributions Where the Bootstrap Fails](https://mlsanity.com/paper/2607.16666/) (2026)
- [Priors learned from legacy reconstructions inherit undetectable overconfidence](https://mlsanity.com/paper/2607.21721/) (2026)
- [Bayesian Uncertainty Quantification for fMRI Functional Connectivity via Simulation-Based Inference](https://mlsanity.com/paper/2609.30445/) (2026)
- [A Missing Latent, Not a Missing Simulator: Radius-Augmented Inference for Real JWST Retrieval](https://mlsanity.com/paper/2610.02245/) (2026)

## Abstract (arXiv)

We present extensive empirical evidence showing that current Bayesian simulation-based inference algorithms can produce computationally unfaithful posterior approximations. Our results show that all benchmarked algorithms -- (Sequential) Neural Posterior Estimation, (Sequential) Neural Ratio Estimation, Sequential Neural Likelihood and variants of Approximate Bayesian Computation -- can yield overconfident posterior approximations, which makes them unreliable for scientific use cases and falsificationist inquiry. Failing to address this issue may reduce the range of applicability of simulation-based inference. For this reason, we argue that research efforts should be made towards theoretical and methodological developments of conservative approximate inference algorithms and present research directions towards this objective. In this regard, we show empirical evidence that ensembling posterior surrogates provides more reliable approximations and mitigates the issue.