# Online multiple testing with e-values

arXiv: https://arxiv.org/abs/2311.06412 · HTML: https://mlsanity.com/paper/2311.06412/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 69 references that are in the map.
- First posted on arXiv 10 Nov 2023.

## Builds on (most cited first, 30 of 36)

- [Time-uniform, nonparametric, nonasymptotic confidence sequences](https://mlsanity.com/paper/1810.08240/) (2018)
- [Estimating means of bounded random variables by betting](https://mlsanity.com/paper/2010.09686/) (2020)
- [Game-theoretic statistics and safe anytime-valid inference](https://mlsanity.com/paper/2210.01948/) (2022)
- [Preserving Statistical Validity in Adaptive Data Analysis](https://mlsanity.com/paper/1411.2664/) (2014)
- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [Kaggle forecasting competitions: An overlooked learning opportunity](https://mlsanity.com/paper/2009.07701/) (2020)
- [Admissible anytime-valid sequential inference must rely on nonnegative martingales](https://mlsanity.com/paper/2009.03167/) (2020)
- [Universal Inference](https://mlsanity.com/paper/1912.11436/) (2019)
- [Model-free selective inference under covariate shift via weighted conformal p-values](https://mlsanity.com/paper/2307.09291/) (2023)
- [Confidence sequences for sampling without replacement](https://mlsanity.com/paper/2006.04347/) (2020)
- [Online Rules for Control of False Discovery Rate and False Discovery Exceedance](https://mlsanity.com/paper/1603.09000/) (2016)
- [A Two-Sample Conditional Distribution Test Using Conformal Prediction and Weighted Rank Sum](https://mlsanity.com/paper/2010.07147/) (2020)
- [A framework for Multi-A(rmed)/B(andit) testing with online FDR control](https://mlsanity.com/paper/1706.05378/) (2017)
- [Post-selection inference for e-value based confidence intervals](https://mlsanity.com/paper/2203.12572/) (2022)
- [Derandomized knockoffs: leveraging e-values for false discovery rate control](https://mlsanity.com/paper/2205.15461/) (2022)
- [Online control of the false discovery rate with decaying memory](https://mlsanity.com/paper/1710.00499/) (2017)
- [SAFFRON: an adaptive algorithm for online control of the false discovery rate](https://mlsanity.com/paper/1802.09098/) (2018)
- [Online Control of the False Coverage Rate and False Sign Rate](https://mlsanity.com/paper/1905.01059/) (2019)
- [ADDIS: an adaptive discarding algorithm for online FDR control with conservative nulls](https://mlsanity.com/paper/1905.11465/) (2019)
- [Asynchronous Online Testing of Multiple Hypotheses](https://mlsanity.com/paper/1812.05068/) (2018)
- [A unified framework for bandit multiple testing](https://mlsanity.com/paper/2107.07322/) (2021)
- [Online control of the familywise error rate](https://mlsanity.com/paper/1910.04900/) (2019)
- [On Online Control of False Discovery Rate](https://mlsanity.com/paper/1502.06197/) (2015)
- [The Power of Batching in Multiple Hypothesis Testing](https://mlsanity.com/paper/1910.04968/) (2019)
- [Universal Inference Meets Random Projections: A Scalable Test for Log-concavity](https://mlsanity.com/paper/2111.09254/) (2021)
- [E-values as unnormalized weights in multiple testing](https://mlsanity.com/paper/2204.12447/) (2022)
- [Near-optimal multiple testing in Bayesian linear models with finite-sample FDR control](https://mlsanity.com/paper/2211.02778/) (2022)
- [More powerful multiple testing under dependence via randomization](https://mlsanity.com/paper/2305.11126/) (2023)
- [Online multiple testing with super-uniformity reward](https://mlsanity.com/paper/2110.01255/) (2021)
- [Online control of the False Discovery Rate in group-sequential platform trials](https://mlsanity.com/paper/2112.10619/) (2021)

## Built on by (most cited first, 12 of 12)

- [Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization](https://mlsanity.com/paper/2411.17983/) (2024)
- [On Continuous Monitoring of Risk Violations under Unknown Shift](https://mlsanity.com/paper/2506.16416/) (2025)
- [Auditing Pay-Per-Token in Large Language Models](https://mlsanity.com/paper/2510.05181/) (2025)
- [Ensuring Reliability via Hyperparameter Selection: Review and Advances](https://mlsanity.com/paper/2502.04206/) (2025)
- [Online Conformal Selection with Accept-to-Reject Changes](https://mlsanity.com/paper/2508.13838/) (2025)
- [Model-agnostic Selective Labeling with Provable Statistical Guarantees](https://mlsanity.com/paper/2510.14581/) (2025)
- [Global Sequential Testing for Multi-Stream Auditing](https://mlsanity.com/paper/2602.21479/) (2026)
- [Optimal Rates for Differentially Private Hypothesis Testing with E-values](https://mlsanity.com/paper/2605.28952/) (2026)
- [Incentive Aware AI Regulations: A Credal Characterisation](https://mlsanity.com/paper/2603.05175/) (2026)
- [A Regret Perspective on Online Multiple Testing](https://mlsanity.com/paper/2605.13916/) (2026)
- [Always-On Experimentation](https://mlsanity.com/paper/2609.38695/) (2026)
- [Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Search](https://mlsanity.com/paper/2610.01751/) (2026)

## Abstract (arXiv)

A scientist tests a continuous stream of hypotheses over time in the course of her investigation -- she does not test a predetermined, fixed number of hypotheses. The scientist wishes to make as many discoveries as possible while ensuring the number of false discoveries is controlled -- a well recognized way for accomplishing this is to control the false discovery rate (FDR). Prior methods for FDR control in the online setting have focused on formulating algorithms when specific dependency structures are assumed to exist between the test statistics of each hypothesis. However, in practice, these dependencies often cannot be known beforehand or tested after the fact. Our algorithm, e-LOND, provides FDR control under arbitrary, possibly unknown, dependence. We show that our method is more powerful than existing approaches to this problem through simulations. We also formulate extensions of this algorithm to utilize randomization for increased power, and for constructing confidence intervals in online selective inference.