mlsanityData to 14 Oct 2026

A Three-Way Testing Framework for Quantifying Epistemic Calibration Uncertainty in SBI

arXiv 2609.24419 ↗First posted 21 Sep 2026stat.ME stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Current experimental scientists increasingly rely on simulation-based inference (SBI) to invert complex models with intractable likelihoods. A primary goal in these settings is to obtain credible regions with valid coverage. While recent model-agnostic conformal calibration methods have succeeded in constructing credible sets with prescribed local Bayesian coverage, their approximate nature introduces inherent epistemic uncertainty in the calibration process. In this work, we propose a novel tool for diagnosing calibration uncertainty. Our approach is based on a simple three-way hypothesis testing procedure. We demonstrate how this tool can be used to effectively assess necessary simulation budgets for calibration sets and analyze the epistemic uncertainty associated with cutoff estimation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 50 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

28 of 50 references.

References · 28

  1. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  2. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  3. 2017Masked Autoregressive Flow for Density Estimationcited by 684
  4. 2019Neural Spline Flowscited by 596
  5. 2019The frontier of simulation-based inferencecited by 502
  6. 2018Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flowscited by 221
  7. 2019Automatic Posterior Transformation for Likelihood-Free Inferencecited by 219
  8. 2017Flexible statistical inference for mechanistic models of neural dynamicscited by 163
  9. 2020On Contrastive Learning for Likelihood-free Inferencecited by 98
  10. 2015Approximating Likelihood Ratios with Calibrated Discriminative Classifierscited by 90
  11. 2015Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Modelscited by 61
  12. 2016Deep Learning in Bioinformaticscited by 48
16 more
  1. 2022Compositional Score Modeling for Simulation-based Inferencecited by 48
  2. 2022Truncated proposals for scalable and hassle-free simulation-based inferencecited by 41
  3. 2025Simulation-Based Inference: A Practical Guidecited by 32
  4. 2014High-Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computationcited by 19
  5. 2021HNPE: Leveraging Global Parameters for Neural Posterior Estimationcited by 19
  6. 2024Diffusion posterior sampling for simulation-based inference in tall data settingscited by 19
  7. 2020Confidence Sets and Hypothesis Testing in a Likelihood-Free Inference Settingcited by 18
  8. 2022Simulator-Based Inference with Waldo: Confidence Regions by Leveraging Prediction Algorithms and Posterior Estimators for Inverse Problemscited by 15
  9. 2024Regression Trees for Fast and Adaptive Prediction Intervalscited by 12
  10. 2021Likelihood-Free Frequentist Inference: Bridging Classical Statistics and Machine Learning for Reliable Simulator-Based Inferencecited by 11
  11. 2025Epistemic Uncertainty in Conformal Scores: A Unified Approachcited by 9
  12. 2018ABC-CDE: Towards Approximate Bayesian Computation with Complex High-Dimensional Data and Limited Simulationscited by 8
  13. 2024Approximate Bayesian Computation with Deep Learning and Conformal predictioncited by 6
  14. 2025CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inferencecited by 6
  15. 2023REACT to NHST: Sensible conclusions to meaningful hypothesescited by 2
  16. 2024Simulation-based Inference of Developmental EEG Maturation with the Spectral Graph Modelcited by 2

Built on by

Not cited yet.