# Asynchronous Online Testing of Multiple Hypotheses

arXiv: https://arxiv.org/abs/1812.05068 · HTML: https://mlsanity.com/paper/1812.05068/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2018.

## Built on by (most cited first, 10 of 10)

- [Online false discovery rate control for anomaly detection in time series](https://mlsanity.com/paper/2112.03196/) (2021)
- [Online multiple testing with e-values](https://mlsanity.com/paper/2311.06412/) (2023)
- [Online Control of the False Discovery Rate under "Decision Deadlines"](https://mlsanity.com/paper/2110.01583/) (2021)
- [Online False Discovery Rate Control for LORD & SAFFRON Under Positive, Local Dependence](https://mlsanity.com/paper/2110.08161/) (2021)
- [Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture](https://mlsanity.com/paper/2511.06701/) (2025)
- [Global Sequential Testing for Multi-Stream Auditing](https://mlsanity.com/paper/2602.21479/) (2026)
- [Cost-aware Generalized α-investing for Multiple Hypothesis Testing](https://mlsanity.com/paper/2210.17514/) (2022)
- [The Query/Hit Model for Sequential Hypothesis Testing](https://mlsanity.com/paper/2502.00605/) (2025)
- [A Regret Perspective on Online Multiple Testing](https://mlsanity.com/paper/2605.13916/) (2026)
- [Always-On Experimentation](https://mlsanity.com/paper/2609.38695/) (2026)

## Abstract (arXiv)

We consider the problem of asynchronous online testing, aimed at providing control of the false discovery rate (FDR) during a continual stream of data collection and testing, where each test may be a sequential test that can start and stop at arbitrary times. This setting increasingly characterizes real-world applications in science and industry, where teams of researchers across large organizations may conduct tests of hypotheses in a decentralized manner. The overlap in time and space also tends to induce dependencies among test statistics, a challenge for classical methodology, which either assumes (overly optimistically) independence or (overly pessimistically) arbitrary dependence between test statistics. We present a general framework that addresses both of these issues via a unified computational abstraction that we refer to as "conflict sets." We show how this framework yields algorithms with formal FDR guarantees under a more intermediate, local notion of dependence. We illustrate our algorithms in simulations by comparing to existing algorithms for online FDR control.