# AdaPT: An interactive procedure for multiple testing with side information

arXiv: https://arxiv.org/abs/1609.06035 · HTML: https://mlsanity.com/paper/1609.06035/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2016.

## Built on by (most cited first, 16 of 16)

- [Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers](https://mlsanity.com/paper/2208.11111/) (2022)
- [Data fission: splitting a single data point](https://mlsanity.com/paper/2112.11079/) (2021)
- [Model-X Sequential Testing for Conditional Independence via Testing by Betting](https://mlsanity.com/paper/2210.00354/) (2022)
- [A Locally Adaptive Algorithm for Multiple Testing with Network Structure](https://mlsanity.com/paper/2203.11461/) (2022)
- [Adaptive False Discovery Rate Control with Privacy Guarantee](https://mlsanity.com/paper/2305.19482/) (2023)
- [Treatment response as a latent variable](https://mlsanity.com/paper/2502.08776/) (2025)
- [Controlling False Discovery in Arbitrarily Structured Hypothesis Spaces via Reproducing Kernels](https://mlsanity.com/paper/2605.17559/) (2026)
- [Controlling False Discovery Rates under Cross-Sectional Correlations](https://mlsanity.com/paper/2102.07826/) (2021)
- [Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals](https://mlsanity.com/paper/2106.08881/) (2021)
- [AdaPT-GMM: Powerful and robust covariate-assisted multiple testing](https://mlsanity.com/paper/2106.15812/) (2021)
- [Error Controlled Feature Selection for Ultrahigh Dimensional and Highly Correlated Feature Space Using Deep Learning](https://mlsanity.com/paper/2209.07011/) (2022)
- [Empirical Bayes Estimation with Side Information: A Nonparametric Integrative Tweedie Approach](https://mlsanity.com/paper/2308.05883/) (2023)
- [A statistical method for crack pre-detection in 3D concrete images](https://mlsanity.com/paper/2402.16126/) (2024)
- [DART2: a robust multiple testing method to smartly leverage helpful or misleading ancillary information](https://mlsanity.com/paper/2409.03618/) (2024)
- [Safe Urban Traffic Control via Uncertainty-Aware Conformal Prediction and World-Model Reinforcement Learning](https://mlsanity.com/paper/2602.04821/) (2026)
- [Finite Resources False Discovery Rate Control in Structured Hypothesis Spaces](https://mlsanity.com/paper/2606.15393/) (2026)

## Abstract (arXiv)

We consider the problem of multiple hypothesis testing with generic side information: for each hypothesis Hi we observe both a p-value pi and some predictor xi encoding contextual information about the hypothesis. For large-scale problems, adaptively focusing power on the more promising hypotheses (those more likely to yield discoveries) can lead to much more powerful multiple testing procedures. We propose a general iterative framework for this problem, called the Adaptive p-value Thresholding (AdaPT) procedure, which adaptively estimates a Bayes-optimal p-value rejection threshold and controls the false discovery rate (FDR) in finite samples. At each iteration of the procedure, the analyst proposes a rejection threshold and observes partially censored p-values, estimates the false discovery proportion (FDP) below the threshold, and either stops to reject or proposes another threshold, until the estimated FDP is below α. Our procedure is adaptive in an unusually strong sense, permitting the analyst to use any statistical or machine learning method she chooses to estimate the optimal threshold, and to switch between different models at each iteration as information accrues. We demonstrate the favorable performance of AdaPT by comparing it to state-of-the-art methods in five real applications and two simulation studies.