# Confidence Sets and Hypothesis Testing in a Likelihood-Free Inference Setting

arXiv: https://arxiv.org/abs/2002.10399 · HTML: https://mlsanity.com/paper/2002.10399/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2020.

## Built on by (most cited first, 18 of 18)

- [A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful](https://mlsanity.com/paper/2110.06581/) (2021)
- [Towards Reliable Simulation-Based Inference with Balanced Neural Ratio Estimation](https://mlsanity.com/paper/2208.13624/) (2022)
- [L-C2ST: Local Diagnostics for Posterior Approximations in Simulation-Based Inference](https://mlsanity.com/paper/2306.03580/) (2023)
- [Likelihood-Free Frequentist Inference: Bridging Classical Statistics and Machine Learning for Reliable Simulator-Based Inference](https://mlsanity.com/paper/2107.03920/) (2021)
- [Balancing Simulation-based Inference for Conservative Posteriors](https://mlsanity.com/paper/2304.10978/) (2023)
- [Neural Likelihood Surfaces for Spatial Processes with Computationally Intensive or Intractable Likelihoods](https://mlsanity.com/paper/2305.04634/) (2023)
- [Amortised Likelihood-free Inference for Expensive Time-series Simulators with Signatured Ratio Estimation](https://mlsanity.com/paper/2202.11585/) (2022)
- [Contrastive Neural Ratio Estimation for Simulation-based Inference](https://mlsanity.com/paper/2210.06170/) (2022)
- [A Kernel-Based Conditional Two-Sample Test Using Nearest Neighbors (with Applications to Calibration, Regression Curves, and Simulation-Based Inference)](https://mlsanity.com/paper/2407.16550/) (2024)
- [CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference](https://mlsanity.com/paper/2508.17077/) (2025)
- [Tests for model misspecification in simulation-based inference: from local distortions to global model checks](https://mlsanity.com/paper/2412.15100/) (2024)
- [Conformal Calibration of Statistical Confidence Sets](https://mlsanity.com/paper/2411.19368/) (2024)
- [Kernel-Based Tests for Likelihood-Free Hypothesis Testing](https://mlsanity.com/paper/2308.09043/) (2023)
- [Trustworthy scientific inference with generative models](https://mlsanity.com/paper/2508.02602/) (2025)
- [Sandbagging in a Simple Survival Bandit Problem](https://mlsanity.com/paper/2509.26239/) (2025)
- [Super-Level-Set Regression: Conditional Quantiles via Volume Minimization](https://mlsanity.com/paper/2605.06210/) (2026)
- [A Three-Way Testing Framework for Quantifying Epistemic Calibration Uncertainty in SBI](https://mlsanity.com/paper/2609.24419/) (2026)
- [SLICE: Sequential Likelihood-Free Inference via Conformal E-Values](https://mlsanity.com/paper/2610.13855/) (2026)

## Abstract (arXiv)

Parameter estimation, statistical tests and confidence sets are the cornerstones of classical statistics that allow scientists to make inferences about the underlying process that generated the observed data. A key question is whether one can still construct hypothesis tests and confidence sets with proper coverage and high power in a so-called likelihood-free inference (LFI) setting; that is, a setting where the likelihood is not explicitly known but one can forward-simulate observable data according to a stochastic model. In this paper, we present textttACORE (Approximate Computation via Odds Ratio Estimation), a frequentist approach to LFI that first formulates the classical likelihood ratio test (LRT) as a parametrized classification problem, and then uses the equivalence of tests and confidence sets to build confidence regions for parameters of interest. We also present a goodness-of-fit procedure for checking whether the constructed tests and confidence regions are valid. textttACORE is based on the key observation that the LRT statistic, the rejection probability of the test, and the coverage of the confidence set are conditional distribution functions which often vary smoothly as a function of the parameters of interest. Hence, instead of relying solely on samples simulated at fixed parameter settings (as is the convention in standard Monte Carlo solutions), one can leverage machine learning tools and data simulated in the neighborhood of a parameter to improve estimates of quantities of interest. We demonstrate the efficacy of textttACORE with both theoretical and empirical results. Our implementation is available on Github.