# Beyond Neyman-Pearson: e-values enable hypothesis testing with a data-driven alpha

arXiv: https://arxiv.org/abs/2205.00901 · HTML: https://mlsanity.com/paper/2205.00901/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2022.

## Built on by (most cited first, 14 of 14)

- [E-Values Expand the Scope of Conformal Prediction](https://mlsanity.com/paper/2503.13050/) (2025)
- [Prediction-Powered E-Values](https://mlsanity.com/paper/2502.04294/) (2025)
- [Backward Conformal Prediction](https://mlsanity.com/paper/2505.13732/) (2025)
- [Adaptive Coverage Policies in Conformal Prediction](https://mlsanity.com/paper/2510.04318/) (2025)
- [Fuzzy Prediction Sets: Conformal Prediction with E-values](https://mlsanity.com/paper/2509.13130/) (2025)
- [E-Scores for (In)Correctness Assessment of Generative Model Outputs](https://mlsanity.com/paper/2510.25770/) (2025)
- [SCOPE: Sequential Conformal Probing for Reliable OOD Rejection in LLM Services](https://mlsanity.com/paper/2606.21255/) (2026)
- [Sequential Structure-Sensitive Residual Diagnostics for PDE Inverse Problems](https://mlsanity.com/paper/2607.02101/) (2026)
- [Valid Inference for Machine Learning Model Parameters](https://mlsanity.com/paper/2302.10840/) (2023)
- [Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction](https://mlsanity.com/paper/2403.19605/) (2024)
- [Improving Backward Conformal Prediction via Non-Conformity Score Transformation](https://mlsanity.com/paper/2602.01733/) (2026)
- [Model-free generalized fiducial inference](https://mlsanity.com/paper/2307.12472/) (2023)
- [Post-Selection Distributional Model Evaluation](https://mlsanity.com/paper/2603.23055/) (2026)
- [Bentkus-type asymptotic e-values](https://mlsanity.com/paper/2606.06332/) (2026)

## Abstract (arXiv)

A standard practice in statistical hypothesis testing is to mention the p-value alongside the accept/reject decision. We show the advantages of mentioning an e-value instead. With p-values, it is not clear how to use an extreme observation (e.g. p ll α) for getting better frequentist decisions. With e-values it is straightforward, since they provide Type-I risk control in a generalized Neyman-Pearson setting with the decision task (a general loss function) determined post-hoc, after observation of the data -- thereby providing a handle on `roving α's'. When Type-II risks are taken into consideration, the only admissible decision rules in the post-hoc setting turn out to be e-value-based. Similarly, if the loss incurred when specifying a faulty confidence interval is not fixed in advance, standard confidence intervals and distributions may fail whereas e-confidence sets and e-posteriors still provide valid risk guarantees. Sufficiently powerful e-values have by now been developed for a range of classical testing problems. We discuss the main challenges for wider development and deployment.