# Online Rules for Control of False Discovery Rate and False Discovery Exceedance

arXiv: https://arxiv.org/abs/1603.09000 · HTML: https://mlsanity.com/paper/1603.09000/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2016.

## Built on by (most cited first, 21 of 21)

- [E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing](https://mlsanity.com/paper/2512.03109/) (2025)
- [Online false discovery rate control for anomaly detection in time series](https://mlsanity.com/paper/2112.03196/) (2021)
- [Online multiple testing with e-values](https://mlsanity.com/paper/2311.06412/) (2023)
- [Online Conformal Selection with Accept-to-Reject Changes](https://mlsanity.com/paper/2508.13838/) (2025)
- [QuTE: decentralized multiple testing on sensor networks with false discovery rate control](https://mlsanity.com/paper/2210.04334/) (2022)
- [Controlling the False Split Rate in Tree-Based Aggregation](https://mlsanity.com/paper/2108.05350/) (2021)
- [Online Control of the False Discovery Rate under "Decision Deadlines"](https://mlsanity.com/paper/2110.01583/) (2021)
- [Online False Discovery Rate Control for LORD & SAFFRON Under Positive, Local Dependence](https://mlsanity.com/paper/2110.08161/) (2021)
- [GRASP: A Goodness-of-Fit Test for Classification Learning](https://mlsanity.com/paper/2209.02064/) (2022)
- [Adaptive Data Analysis for Growing Data](https://mlsanity.com/paper/2405.13375/) (2024)
- [Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture](https://mlsanity.com/paper/2511.06701/) (2025)
- [Global Sequential Testing for Multi-Stream Auditing](https://mlsanity.com/paper/2602.21479/) (2026)
- [Conformal Anomaly Detection in Python: Moving Beyond Heuristic Thresholds with nonconform](https://mlsanity.com/paper/2605.13642/) (2026)
- [Cost-aware Generalized α-investing for Multiple Hypothesis Testing](https://mlsanity.com/paper/2210.17514/) (2022)
- [Online Conformal Anomaly Detection with Prediction-Powered Data Acquisition](https://mlsanity.com/paper/2505.01783/) (2025)
- [Online Reliable Anomaly Detection via Neuromorphic Sensing and Communications](https://mlsanity.com/paper/2510.14688/) (2025)
- [Safe Urban Traffic Control via Uncertainty-Aware Conformal Prediction and World-Model Reinforcement Learning](https://mlsanity.com/paper/2602.04821/) (2026)
- [A Regret Perspective on Online Multiple Testing](https://mlsanity.com/paper/2605.13916/) (2026)
- [Distributional Granger Causality: Identification, Sequential Inference, and Adaptive Testing](https://mlsanity.com/paper/2606.22230/) (2026)
- [Which Self-Improvements Should We Trust? Reliable Self-Improvement When Agents Reuse Their Benchmarks](https://mlsanity.com/paper/2609.33180/) (2026)
- [Always-On Experimentation](https://mlsanity.com/paper/2609.38695/) (2026)

## Abstract (arXiv)

Multiple hypothesis testing is a core problem in statistical inference and arises in almost every scientific field. Given a set of null hypotheses H(n) = (H1,dotsc, Hn), Benjamini and Hochberg introduced the false discovery rate (FDR), which is the expected proportion of false positives among rejected null hypotheses, and proposed a testing procedure that controls FDR below a pre-assigned significance level. Nowadays FDR is the criterion of choice for large scale multiple hypothesis testing. In this paper we consider the problem of controlling FDR in an "online manner". Concretely, we consider an ordered --possibly infinite-- sequence of null hypotheses H = (H1,H2,H3,dots ) where, at each step i, the statistician must decide whether to reject hypothesis Hi having access only to the previous decisions. This model was introduced by Foster and Stine. We study a class of "generalized alpha-investing" procedures and prove that any rule in this class controls online FDR, provided p-values corresponding to true nulls are independent from the other p-values. (Earlier work only established mFDR control.) Next, we obtain conditions under which generalized alpha-investing controls FDR in the presence of general p-values dependencies. Finally, we develop a modified set of procedures that also allow to control the false discovery exceedance (the tail of the proportion of false discoveries). Numerical simulations and analytical results indicate that online procedures do not incur a large loss in statistical power with respect to offline approaches, such as Benjamini-Hochberg.