# BONuS: Multiple multivariate testing with a data-adaptivetest statistic

arXiv: https://arxiv.org/abs/2106.15743 · HTML: https://mlsanity.com/paper/2106.15743/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2021.

## Built on by (most cited first, 3 of 3)

- [Conformal link prediction for false discovery rate control](https://mlsanity.com/paper/2306.14693/) (2023)
- [On the Adversarial Robustness of Learning-based Conformal Novelty Detection](https://mlsanity.com/paper/2510.00463/) (2025)
- [Structure-Adaptive Conformal Inference for Large-Scale Out-of-Distribution Testing](https://mlsanity.com/paper/2605.26429/) (2026)

## Abstract (arXiv)

We propose a new adaptive empirical Bayes framework, the Bag-Of-Null-Statistics (BONuS) procedure, for multiple testing where each hypothesis testing problem is itself multivariate or nonparametric. BONuS is an adaptive and interactive knockoff-type method that helps improve the testing power while controlling the false discovery rate (FDR), and is closely connected to the "counting knockoffs" procedure analyzed in Weinstein et al. (2017). Contrary to procedures that start with a p-value for each hypothesis, our method analyzes the entire data set to adaptively estimate an optimal p-value transform based on an empirical Bayes model. Despite the extra adaptivity, our method controls FDR in finite samples even if the empirical Bayes model is incorrect or the estimation is poor. An extension, the Double BONuS procedure, validates the empirical Bayes model to guard against power loss due to model misspecification.