# Conformal Calibration of Statistical Confidence Sets

arXiv: https://arxiv.org/abs/2411.19368 · HTML: https://mlsanity.com/paper/2411.19368/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 67 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Builds on (most cited first, 30 of 30)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Automatic Posterior Transformation for Likelihood-Free Inference](https://mlsanity.com/paper/1905.07488/) (2019)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction](https://mlsanity.com/paper/2106.08460/) (2021)
- [Class-Conditional Conformal Prediction with Many Classes](https://mlsanity.com/paper/2306.09335/) (2023)
- [CD-split and HPD-split: efficient conformal regions in high dimensions](https://mlsanity.com/paper/2007.12778/) (2020)
- [Flexible distribution-free conditional predictive bands using density estimators](https://mlsanity.com/paper/1910.05575/) (2019)
- [Likelihood-free inference with emulator networks](https://mlsanity.com/paper/1805.09294/) (2018)
- [Mining gold from implicit models to improve likelihood-free inference](https://mlsanity.com/paper/1805.12244/) (2018)
- [normflows: A PyTorch Package for Normalizing Flows](https://mlsanity.com/paper/2302.12014/) (2023)
- [Conformal prediction with local weights: randomization enables local guarantees](https://mlsanity.com/paper/2310.07850/) (2023)
- [Criteria of efficiency for conformal prediction](https://mlsanity.com/paper/1603.04416/) (2016)
- [Diagnostics for Conditional Density Models and Bayesian Inference Algorithms](https://mlsanity.com/paper/2102.10473/) (2021)
- [A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful](https://mlsanity.com/paper/2110.06581/) (2021)
- [Computationally efficient versions of conformal predictive distributions](https://mlsanity.com/paper/1911.00941/) (2019)
- [High-Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computation](https://mlsanity.com/paper/1404.7063/) (2014)
- [Confidence Sets and Hypothesis Testing in a Likelihood-Free Inference Setting](https://mlsanity.com/paper/2002.10399/) (2020)
- [Simulator-Based Inference with Waldo: Confidence Regions by Leveraging Prediction Algorithms and Posterior Estimators for Inverse Problems](https://mlsanity.com/paper/2205.15680/) (2022)
- [Regression Trees for Fast and Adaptive Prediction Intervals](https://mlsanity.com/paper/2402.07357/) (2024)
- [Likelihood-Free Frequentist Inference: Bridging Classical Statistics and Machine Learning for Reliable Simulator-Based Inference](https://mlsanity.com/paper/2107.03920/) (2021)
- [ABC-CDE: Towards Approximate Bayesian Computation with Complex High-Dimensional Data and Limited Simulations](https://mlsanity.com/paper/1805.05480/) (2018)
- [Distribution-Free Conformal Joint Prediction Regions for Neural Marked Temporal Point Processes](https://mlsanity.com/paper/2401.04612/) (2024)
- [Approximate Bayesian Computation with Deep Learning and Conformal prediction](https://mlsanity.com/paper/2406.04874/) (2024)
- [Conformal Predictive Systems Under Covariate Shift](https://mlsanity.com/paper/2404.15018/) (2024)
- [From conformal to probabilistic prediction](https://mlsanity.com/paper/1406.5600/) (2014)
- [REACT to NHST: Sensible conclusions to meaningful hypotheses](https://mlsanity.com/paper/2308.09112/) (2023)

## Built on by (most cited first, 3 of 3)

- [Epistemic Uncertainty in Conformal Scores: A Unified Approach](https://mlsanity.com/paper/2502.06995/) (2025)
- [LoBoost: Fast Model-Native Local Conformal Prediction for Gradient-Boosted Trees](https://mlsanity.com/paper/2602.22432/) (2026)
- [SLICE: Sequential Likelihood-Free Inference via Conformal E-Values](https://mlsanity.com/paper/2610.13855/) (2026)

## Abstract (arXiv)

Constructing valid confidence sets is a crucial task in statistical inference, yet traditional methods often face challenges when dealing with complex models or limited observed sample sizes. These challenges are frequently encountered in modern applications, such as Likelihood-Free Inference (LFI). In these settings, confidence sets may fail to maintain a confidence level close to the nominal value. In this paper, we introduce two novel methods, TRUST and TRUST++, for calibrating confidence sets to achieve distribution-free conditional coverage. These methods rely entirely on simulated data from the statistical model to perform calibration. Leveraging insights from conformal prediction techniques adapted to the statistical inference context, our methods ensure both finite-sample local coverage and asymptotic conditional coverage as the number of simulations increases, even if n is small. They effectively handle nuisance parameters and provide computationally efficient uncertainty quantification for the estimated confidence sets. This allows users to assess whether additional simulations are necessary for robust inference. Through theoretical analysis and experiments on models with both tractable and intractable likelihoods, we demonstrate that our methods outperform existing approaches, particularly in small-sample regimes. This work bridges the gap between conformal prediction and statistical inference, offering practical tools for constructing valid confidence sets in complex models.