# Simulator-Based Inference with Waldo: Confidence Regions by Leveraging Prediction Algorithms and Posterior Estimators for Inverse Problems

arXiv: https://arxiv.org/abs/2205.15680 · HTML: https://mlsanity.com/paper/2205.15680/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 50 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 13 of 13)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [Distributional conformal prediction](https://mlsanity.com/paper/1909.07889/) (2019)
- [Sampling using SU(N) gauge equivariant flows](https://mlsanity.com/paper/2008.05456/) (2020)
- [Mining gold from implicit models to improve likelihood-free inference](https://mlsanity.com/paper/1805.12244/) (2018)
- [Fast covariance parameter estimation of spatial Gaussian process models using neural networks](https://mlsanity.com/paper/2012.15339/) (2020)
- [A neural simulation-based inference approach for characterizing the Galactic Center γ-ray excess](https://mlsanity.com/paper/2110.06931/) (2021)
- [Adaptive Approximate Bayesian Computation Tolerance Selection](https://mlsanity.com/paper/1907.01505/) (2019)
- [Component-wise approximate Bayesian computation via Gibbs-like steps](https://mlsanity.com/paper/1905.13599/) (2019)

## Built on by (most cited first, 15 of 15)

- [Neural Methods for Amortized Inference](https://mlsanity.com/paper/2404.12484/) (2024)
- [Semi-Supervised Risk Control via Prediction-Powered Inference](https://mlsanity.com/paper/2412.11174/) (2024)
- [Epistemic Uncertainty in Conformal Scores: A Unified Approach](https://mlsanity.com/paper/2502.06995/) (2025)
- [CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference](https://mlsanity.com/paper/2508.17077/) (2025)
- [Low-Budget Simulation-Based Inference with Bayesian Neural Networks](https://mlsanity.com/paper/2408.15136/) (2024)
- [Conformal Calibration of Statistical Confidence Sets](https://mlsanity.com/paper/2411.19368/) (2024)
- [Meta-Analysis with Untrusted Data](https://mlsanity.com/paper/2407.09387/) (2024)
- [Classification under Nuisance Parameters and Generalized Label Shift in Likelihood-Free Inference](https://mlsanity.com/paper/2402.05330/) (2024)
- [Sequential Kernelized Stein Discrepancy](https://mlsanity.com/paper/2409.17505/) (2024)
- [Trustworthy scientific inference with generative models](https://mlsanity.com/paper/2508.02602/) (2025)
- [MIRA: A Score for Conditional Distribution Accuracy and Model Comparison](https://mlsanity.com/paper/2605.02014/) (2026)
- [Conservative neural posterior estimation via distributionally robust training](https://mlsanity.com/paper/2605.28516/) (2026)
- [Beyond Field Accuracy: Two-Axis Diagnosis of Inverse-PINN Parameter Error](https://mlsanity.com/paper/2608.15373/) (2026)
- [A Three-Way Testing Framework for Quantifying Epistemic Calibration Uncertainty in SBI](https://mlsanity.com/paper/2609.24419/) (2026)
- [SLICE: Sequential Likelihood-Free Inference via Conformal E-Values](https://mlsanity.com/paper/2610.13855/) (2026)

## Abstract (arXiv)

Prediction algorithms, such as deep neural networks (DNNs), are used in many domain sciences to directly estimate internal parameters of interest in simulator-based models, especially in settings where the observations include images or complex high-dimensional data. In parallel, modern neural density estimators, such as normalizing flows, are becoming increasingly popular for uncertainty quantification, especially when both parameters and observations are high-dimensional. However, parameter inference is an inverse problem and not a prediction task; thus, an open challenge is to construct conditionally valid and precise confidence regions, with a guaranteed probability of covering the true parameters of the data-generating process, no matter what the (unknown) parameter values are, and without relying on large-sample theory. Many simulator-based inference (SBI) methods are indeed known to produce biased or overly confident parameter regions, yielding misleading uncertainty estimates. This paper presents WALDO, a novel method to construct confidence regions with finite-sample conditional validity by leveraging prediction algorithms or posterior estimators that are currently widely adopted in SBI. WALDO reframes the well-known Wald test statistic, and uses a computationally efficient regression-based machinery for classical Neyman inversion of hypothesis tests. We apply our method to a recent high-energy physics problem, where prediction with DNNs has previously led to estimates with prediction bias. We also illustrate how our approach can correct overly confident posterior regions computed with normalizing flows.