# ADDIS: an adaptive discarding algorithm for online FDR control with conservative nulls

arXiv: https://arxiv.org/abs/1905.11465 · HTML: https://mlsanity.com/paper/1905.11465/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 11 of 11)

- [E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing](https://mlsanity.com/paper/2512.03109/) (2025)
- [Online false discovery rate control for anomaly detection in time series](https://mlsanity.com/paper/2112.03196/) (2021)
- [Online multiple testing with e-values](https://mlsanity.com/paper/2311.06412/) (2023)
- [Adaptive False Discovery Rate Control with Privacy Guarantee](https://mlsanity.com/paper/2305.19482/) (2023)
- [Online Control of the False Discovery Rate under "Decision Deadlines"](https://mlsanity.com/paper/2110.01583/) (2021)
- [Online False Discovery Rate Control for LORD & SAFFRON Under Positive, Local Dependence](https://mlsanity.com/paper/2110.08161/) (2021)
- [AdaPT-GMM: Powerful and robust covariate-assisted multiple testing](https://mlsanity.com/paper/2106.15812/) (2021)
- [Online Conformal Anomaly Detection with Prediction-Powered Data Acquisition](https://mlsanity.com/paper/2505.01783/) (2025)
- [Online Selective Conformal Prediction with Asymmetric Rules: A Permutation Test Approach](https://mlsanity.com/paper/2602.10018/) (2026)
- [A Regret Perspective on Online Multiple Testing](https://mlsanity.com/paper/2605.13916/) (2026)
- [Always-On Experimentation](https://mlsanity.com/paper/2609.38695/) (2026)

## Abstract (arXiv)

Major internet companies routinely perform tens of thousands of A/B tests each year. Such large-scale sequential experimentation has resulted in a recent spurt of new algorithms that can provably control the false discovery rate (FDR) in a fully online fashion. However, current state-of-the-art adaptive algorithms can suffer from a significant loss in power if null p-values are conservative (stochastically larger than the uniform distribution), a situation that occurs frequently in practice. In this work, we introduce a new adaptive discarding method called ADDIS that provably controls the FDR and achieves the best of both worlds: it enjoys appreciable power increase over all existing methods if nulls are conservative (the practical case), and rarely loses power if nulls are exactly uniformly distributed (the ideal case). We provide several practical insights on robust choices of tuning parameters, and extend the idea to asynchronous and offline settings as well.