# NxN E-valuation: Hypothesis Certification via a Conformal CRT Null

arXiv: https://arxiv.org/abs/2608.06621 · HTML: https://mlsanity.com/paper/2608.06621/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 20 of 20)

- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs](https://mlsanity.com/paper/2406.01297/) (2024)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Estimating means of bounded random variables by betting](https://mlsanity.com/paper/2010.09686/) (2020)
- [Game-theoretic statistics and safe anytime-valid inference](https://mlsanity.com/paper/2210.01948/) (2022)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [The Unfairness of Popularity Bias in Recommendation](https://mlsanity.com/paper/1907.13286/) (2019)
- [There are natural scores: Full comment on Shafer, "Testing by betting: A strategy for statistical and scientific communication"](https://mlsanity.com/paper/2102.05569/) (2021)
- [Testing randomness](https://mlsanity.com/paper/1906.09256/) (2019)
- [A Survey on Hypothesis Generation for Scientific Discovery in the Era of Large Language Models](https://mlsanity.com/paper/2504.05496/) (2025)
- [The Simpson's Paradox in the Offline Evaluation of Recommendation Systems](https://mlsanity.com/paper/2104.08912/) (2021)
- [Model-X Sequential Testing for Conditional Independence via Testing by Betting](https://mlsanity.com/paper/2210.00354/) (2022)
- [Measuring Evidence against Exchangeability and Group Invariance with E-values](https://mlsanity.com/paper/2310.01153/) (2023)
- [Fast and Memory-Efficient Significant Pattern Mining via Permutation Testing](https://mlsanity.com/paper/1502.04315/) (2015)
- [E-Statistics, Group Invariance and Anytime Valid Testing](https://mlsanity.com/paper/2208.07610/) (2022)
- [Learning Representations for Independence Testing](https://mlsanity.com/paper/2409.06890/) (2024)

## Abstract (arXiv)

We propose NxN E-valuation, a handy, e-value-based hypothesis-certification algorithm that lets a hypothesis be verified without building any case-specific certification procedure---such as constructing a dedicated null hypothesis---as long as a large enough dataset is available. The method is especially suited to LLM-based exploration systems, where LLMs are remarkably good at proposing hypotheses but suffer badly from hallucination; this hallucination prevents us from harvesting LLM outputs directly, and existing remedies each fall short. The most common solutions include letting the LLM verify or correct itself circular verification and held-out testing (where false hypotheses can still pass via spurious correlations), among other remedies detailed in the introduction. To resolve this, NxN E-valuation exploits the naturally existing large training set and lets different samples serve as null hypotheses for one another. This design directly realizes a conditional randomization test (CRT) that certifies each hypothesis. The approach can be a universally better replacement for at least LLM circular verification and held-out-data testing, provided the LLM's generations are hypotheses that apply to each individual sample.