# Anytime-valid simulation-based hypothesis testing

arXiv: https://arxiv.org/abs/2610.08210 · HTML: https://mlsanity.com/paper/2610.08210/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 64 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Builds on (most cited first, 7 of 7)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Approximating Likelihood Ratios with Calibrated Discriminative Classifiers](https://mlsanity.com/paper/1506.02169/) (2015)
- [On the Effectiveness of Hybrid Mutual Information Estimation](https://mlsanity.com/paper/2306.00608/) (2023)
- [Post-hoc α Hypothesis Testing and the Post-hoc p-value](https://mlsanity.com/paper/2312.08040/) (2023)
- [Besag-Clifford e-values for unnormalized testing](https://mlsanity.com/paper/2603.15845/) (2026)
- [The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent](https://mlsanity.com/paper/2609.27765/) (2026)
- [Type-II Error Bounds for Test Supermartingales from Lower-Tail Hypotheses](https://mlsanity.com/paper/2609.27766/) (2026)

## Built on by (most cited first, 1 of 1)

- [SLICE: Sequential Likelihood-Free Inference via Conformal E-Values](https://mlsanity.com/paper/2610.13855/) (2026)

## Abstract (arXiv)

For a given data distribution (Xt)_t ∈ mathbbN ~ Q i.i.d., we investigate the hypothesis testing problem: H0: Q = P0 vs. H1: Q = P1, for two different model probability distributions P0 and P1. In contrast to the standard setting, where analytic densities p0 and p1 are given, here, we consider the density-free setting, where we only have access to i.i.d. simulations (Z0t)_t ∈ mathbbN ~ P0 and (Z1t)_t ∈ mathbbN ~ P1. For this simulation-based hypothesis testing setting, we construct an e-test martingale, resulting in a sequential test with anytime-valid type-I error guarantees, approximate growth optimality, geometrically decaying type-II error bounds, and asymptotic power one. Most ingredients used in our constructions are variants of well known concepts. The value of this paper lies in the compact presentation of an effective, anytime-valid solution for the density-free simulation-based sequential hypothesis testing case.