# Semi-Supervised Hypothesis Testing by Betting on Predictions

arXiv: https://arxiv.org/abs/2605.28533 · HTML: https://mlsanity.com/paper/2605.28533/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 19 of 19)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [PPI++: Efficient Prediction-Powered Inference](https://mlsanity.com/paper/2311.01453/) (2023)
- [Hypothesis testing with e-values](https://mlsanity.com/paper/2410.23614/) (2024)
- [There are natural scores: Full comment on Shafer, "Testing by betting: A strategy for statistical and scientific communication"](https://mlsanity.com/paper/2102.05569/) (2021)
- [Universal Inference](https://mlsanity.com/paper/1912.11436/) (2019)
- [Nonparametric Two-Sample Testing by Betting](https://mlsanity.com/paper/2112.09162/) (2021)
- [Prediction-Powered E-Values](https://mlsanity.com/paper/2502.04294/) (2025)
- [Semi-Supervised Risk Control via Prediction-Powered Inference](https://mlsanity.com/paper/2412.11174/) (2024)
- [Instance-Log-Optimality of Portfolio-Based E-Processes and their Sequential Hypothesis Tests](https://mlsanity.com/paper/2504.02818/) (2025)
- [Anytime Valid Tests of Conditional Independence Under Model-X](https://mlsanity.com/paper/2209.12637/) (2022)
- [Anytime-valid, Bayes-assisted, Prediction-Powered Inference](https://mlsanity.com/paper/2505.18000/) (2025)
- [Exact Anytime-valid Confidence Intervals for Contingency Tables and Beyond](https://mlsanity.com/paper/2203.09785/) (2022)

## Built on by (most cited first, 1 of 1)

- [Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation](https://mlsanity.com/paper/2609.28859/) (2026)

## Abstract (arXiv)

We introduce a testing-by-betting framework that leverages predictions on unlabeled data to enhance the power of sequential hypothesis testing. Given limited samples from the joint distribution of (X,Y), and additional unlabeled samples from the marginal of X, we ask how unlabeled data can be used to hypothesize about the distribution of Y, and the conditional distribution of Ymid X. We introduce an e-statistic and use it to construct a sequential test. Under standard distributional assumptions -- label shift or concept shift -- we establish that the test is anytime valid. Furthermore, we show that for binary data, the e-statistic has non-trivial power. Crucially, our approach retains these properties even when the underlying predictions are inaccurate. Through simulations and applications to large language models evaluation, we demonstrate power gains over baseline approaches, including prediction-powered inference. These gains persist even with relatively limited unlabeled data and when predictions have low accuracy due to weak correlation between X and Y.