# Choosing Between Methods of Combining p-values

arXiv: https://arxiv.org/abs/1707.06897 · HTML: https://mlsanity.com/paper/1707.06897/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2017.

## Built on by (most cited first, 17 of 17)

- [Conformal prediction under ambiguous ground truth](https://mlsanity.com/paper/2307.09302/) (2023)
- [Context-Aware Membership Inference Attacks against Pre-trained Large Language Models](https://mlsanity.com/paper/2409.13745/) (2024)
- [Online false discovery rate control for anomaly detection in time series](https://mlsanity.com/paper/2112.03196/) (2021)
- [A Causal Approach to Detecting Multivariate Time-series Anomalies and Root Causes](https://mlsanity.com/paper/2206.15033/) (2022)
- [Multiple testing for signal-agnostic searches of new physics with machine learning](https://mlsanity.com/paper/2408.12296/) (2024)
- [SHAP-XRT: The Shapley Value Meets Conditional Independence Testing](https://mlsanity.com/paper/2207.07038/) (2022)
- [Efficient Ensemble Conditional Independence Test Framework for Causal Discovery](https://mlsanity.com/paper/2509.21021/) (2025)
- [The Lifecycle of a Statistical Model: Model Failure Detection, Identification, and Refitting](https://mlsanity.com/paper/2202.04166/) (2022)
- [Powerful batch conformal prediction for classification](https://mlsanity.com/paper/2411.02239/) (2024)
- [Adaptive Conformal Anomaly Detection with Time Series Foundation Models for Signal Monitoring](https://mlsanity.com/paper/2604.20122/) (2026)
- [Falsification of Unconfoundedness by Testing Independence of Causal Mechanisms](https://mlsanity.com/paper/2502.06231/) (2025)
- [Window-Based Distribution Shift Detection for Deep Neural Networks](https://mlsanity.com/paper/2210.10897/) (2022)
- [Federated Epidemic Surveillance](https://mlsanity.com/paper/2307.02616/) (2023)
- [Combine and Conquer: A Meta-Analysis on Data Shift and Out-of-Distribution Detection](https://mlsanity.com/paper/2406.16045/) (2024)
- [Understanding Activation Patterns in Artificial Neural Networks by Exploring Stochastic Processes](https://mlsanity.com/paper/2308.00858/) (2023)
- [Data-light Uncertainty Set Merging with Admissibility](https://mlsanity.com/paper/2410.12201/) (2024)
- [Falsifying Causal Graphs With Outlier Events](https://mlsanity.com/paper/2607.12145/) (2026)

## Abstract (arXiv)

Combining p-values from independent statistical tests is a popular approach to meta-analysis, particularly when the data underlying the tests are either no longer available or are difficult to combine. A diverse range of p-value combination methods appear in the literature, each with different statistical properties. Yet all too often the final choice used in a meta-analysis can appear arbitrary, as if all effort has been expended building the models that gave rise to the p-values. Birnbaum (1954) showed that any reasonable p-value combiner must be optimal against some alternative hypothesis. Starting from this perspective and recasting each method of combining p-values as a likelihood ratio test, we present theoretical results for some of the standard combiners which provide guidance about how a powerful combiner might be chosen in practice.