# Online control of the familywise error rate

arXiv: https://arxiv.org/abs/1910.04900 · HTML: https://mlsanity.com/paper/1910.04900/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [Model-X Sequential Testing for Conditional Independence via Testing by Betting](https://mlsanity.com/paper/2210.00354/) (2022)
- [Online multiple testing with e-values](https://mlsanity.com/paper/2311.06412/) (2023)
- [Adaptive Identification of Populations with Treatment Benefit in Clinical Trials: Machine Learning Challenges and Solutions](https://mlsanity.com/paper/2208.05844/) (2022)
- [Online Control of the False Discovery Rate under "Decision Deadlines"](https://mlsanity.com/paper/2110.01583/) (2021)
- [Online False Discovery Rate Control for LORD & SAFFRON Under Positive, Local Dependence](https://mlsanity.com/paper/2110.08161/) (2021)
- [A Regret Perspective on Online Multiple Testing](https://mlsanity.com/paper/2605.13916/) (2026)
- [Which Self-Improvements Should We Trust? Reliable Self-Improvement When Agents Reuse Their Benchmarks](https://mlsanity.com/paper/2609.33180/) (2026)

## Abstract (arXiv)

Biological research often involves testing a growing number of null hypotheses as new data is accumulated over time. We study the problem of online control of the familywise error rate (FWER), that is testing an apriori unbounded sequence of hypotheses (p-values) one by one over time without knowing the future, such that with high probability there are no false discoveries in the entire sequence. This paper unifies algorithmic concepts developed for offline (single batch) FWER control and online false discovery rate control to develop novel online FWER control methods. Though many offline FWER methods (e.g. Bonferroni, fallback procedures and Sidak's method) can trivially be extended to the online setting, our main contribution is the design of new, powerful, adaptive online algorithms that control the FWER when the p-values are independent or locally dependent in time. Our experiments demonstrate substantial gains in power, that are also formally proved in a Gaussian sequence model. Multiple testing, FWER control, online setting.