Online multiple testing with e-values
Not on the map. Why?
Abstract (arXiv)
A scientist tests a continuous stream of hypotheses over time in the course of her investigation -- she does not test a predetermined, fixed number of hypotheses. The scientist wishes to make as many discoveries as possible while ensuring the number of false discoveries is controlled -- a well recognized way for accomplishing this is to control the false discovery rate (FDR). Prior methods for FDR control in the online setting have focused on formulating algorithms when specific dependency structures are assumed to exist between the test statistics of each hypothesis. However, in practice, these dependencies often cannot be known beforehand or tested after the fact. Our algorithm, e-LOND, provides FDR control under arbitrary, possibly unknown, dependence. We show that our method is more powerful than existing approaches to this problem through simulations. We also formulate extensions of this algorithm to utilize randomization for increased power, and for constructing confidence intervals in online selective inference.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 36 of its 69 references that are in the map. First posted on arXiv 10 Nov 2023.
Builds on
36 of 69 references.
References · 36
- 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 229
- 2020Estimating means of bounded random variables by bettingcited by 169
- 2022Game-theoretic statistics and safe anytime-valid inferencecited by 155
- 2014Preserving Statistical Validity in Adaptive Data Analysiscited by 103
- 2019Safe Testingcited by 101
- 2020Kaggle forecasting competitions: An overlooked learning opportunitycited by 56
- 2020Admissible anytime-valid sequential inference must rely on nonnegative martingalescited by 48
- 2019Universal Inferencecited by 44
- 2023Model-free selective inference under covariate shift via weighted conformal p-valuescited by 29
- 2020Confidence sequences for sampling without replacementcited by 23
- 2016Online Rules for Control of False Discovery Rate and False Discovery Exceedancecited by 21
- 2020A Two-Sample Conditional Distribution Test Using Conformal Prediction and Weighted Rank Sumcited by 19
24 more
- 2017A framework for Multi-A(rmed)/B(andit) testing with online FDR controlcited by 17
- 2022Post-selection inference for e-value based confidence intervalscited by 14
- 2022Derandomized knockoffs: leveraging e-values for false discovery rate controlcited by 12
- 2017Online control of the false discovery rate with decaying memorycited by 11
- 2018SAFFRON: an adaptive algorithm for online control of the false discovery ratecited by 11
- 2019Online Control of the False Coverage Rate and False Sign Ratecited by 11
- 2019ADDIS: an adaptive discarding algorithm for online FDR control with conservative nullscited by 11
- 2018Asynchronous Online Testing of Multiple Hypothesescited by 10
- 2021A unified framework for bandit multiple testingcited by 8
- 2019Online control of the familywise error ratecited by 7
- 2015On Online Control of False Discovery Ratecited by 6
- 2019The Power of Batching in Multiple Hypothesis Testingcited by 4
- 2021Universal Inference Meets Random Projections: A Scalable Test for Log-concavitycited by 3
- 2022E-values as unnormalized weights in multiple testingcited by 3
- 2022Near-optimal multiple testing in Bayesian linear models with finite-sample FDR controlcited by 3
- 2023More powerful multiple testing under dependence via randomizationcited by 3
- 2021Online multiple testing with super-uniformity rewardcited by 2
- 2021Online control of the False Discovery Rate in group-sequential platform trialscited by 2
- 2018Online control of the false discovery rate in biomedical researchcited by 1
- 2021Online Control of the False Discovery Rate under "Decision Deadlines"cited by 1
- 2021Online False Discovery Rate Control for LORD & SAFFRON Under Positive, Local Dependencecited by 1
- 2022Online error control for platform trialscited by 1
- 2023A Sequential Test for Log-Concavitycited by 1
- 2023An exhaustive ADDIS principle for online FWER controlcited by 1
Built on by
Citing papers per year: 2024: 1; 2025: 5; 2026: 6.
12 papers.
- 2024Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimizationcited by 14
- 2025On Continuous Monitoring of Risk Violations under Unknown Shiftcited by 10
- 2025Auditing Pay-Per-Token in Large Language Modelscited by 4
- 2025Ensuring Reliability via Hyperparameter Selection: Review and Advancescited by 3
- 2025Online Conformal Selection with Accept-to-Reject Changescited by 3
- 2025Model-agnostic Selective Labeling with Provable Statistical Guaranteescited by 3
- 2026Global Sequential Testing for Multi-Stream Auditingcited by 1
- 2026Optimal Rates for Differentially Private Hypothesis Testing with E-valuescited by 1
- 2026Incentive Aware AI Regulations: A Credal Characterisationcited by 0
- 2026A Regret Perspective on Online Multiple Testingcited by 0
- 2026Always-On Experimentationcited by 0
- 2026Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Searchcited by 0
- 2026Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Search1 Oct 2026
- 2026Always-On Experimentation30 Sep 2026
- 2026Optimal Rates for Differentially Private Hypothesis Testing with E-values27 May 2026
- 2026A Regret Perspective on Online Multiple Testing13 May 2026
- 2026Incentive Aware AI Regulations: A Credal Characterisation5 Mar 2026
- 2026Global Sequential Testing for Multi-Stream Auditing25 Feb 2026
- 2025Model-agnostic Selective Labeling with Provable Statistical Guarantees16 Oct 2025
- 2025Auditing Pay-Per-Token in Large Language Models5 Oct 2025
- 2025Online Conformal Selection with Accept-to-Reject Changes19 Aug 2025
- 2025On Continuous Monitoring of Risk Violations under Unknown Shift19 Jun 2025
- 2025Ensuring Reliability via Hyperparameter Selection: Review and Advances6 Feb 2025
- 2024Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization27 Nov 2024