# Rank-transformed subsampling: inference for multiple data splitting and exchangeable p-values

arXiv: https://arxiv.org/abs/2301.02739 · HTML: https://mlsanity.com/paper/2301.02739/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2023.

## Built on by (most cited first, 5 of 5)

- [Adaptive debiased machine learning using data-driven model selection techniques](https://mlsanity.com/paper/2307.12544/) (2023)
- [Locally minimax optimal confidence sets for the best model](https://mlsanity.com/paper/2503.21639/) (2025)
- [The Debiased Score Test: Hunt-and-test for Semiparametric Hypotheses](https://mlsanity.com/paper/2607.28861/) (2026)
- [AICO: Feature Significance Tests for Supervised Learning](https://mlsanity.com/paper/2506.23396/) (2025)
- [On computing and the complexity of computing higher-order U-statistics, exactly](https://mlsanity.com/paper/2508.12627/) (2025)

## Abstract (arXiv)

Many testing problems are readily amenable to randomised tests such as those employing data splitting. However despite their usefulness in principle, randomised tests have obvious drawbacks. Firstly, two analyses of the same dataset may lead to different results. Secondly, the test typically loses power because it does not fully utilise the entire sample. As a remedy to these drawbacks, we study how to combine the test statistics or p-values resulting from multiple random realisations such as through random data splits. We develop rank-transformed subsampling as a general method for delivering large sample inference about the combined statistic or p-value under mild assumptions. We apply our methodology to a wide range of problems, including testing unimodality in high-dimensional data, testing goodness-of-fit of parametric quantile regression models, testing no direct effect in a sequentially randomised trial and calibrating cross-fit double machine learning confidence intervals. In contrast to existing p-value aggregation schemes that can be highly conservative, our method enjoys type-I error control that asymptotically approaches the nominal level. Moreover, compared to using the ordinary subsampling, we show that our rank transform can remove the first-order bias in approximating the null under alternatives and greatly improve power.