# Multiple testing for signal-agnostic searches of new physics with machine learning

arXiv: https://arxiv.org/abs/2408.12296 · HTML: https://mlsanity.com/paper/2408.12296/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 22 Aug 2024.

## Builds on (most cited first, 8 of 8)

- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Kernel methods through the roof: handling billions of points efficiently](https://mlsanity.com/paper/2006.10350/) (2020)
- [MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting](https://mlsanity.com/paper/2306.08777/) (2023)
- [Machine Learning for Anomaly Detection in Particle Physics](https://mlsanity.com/paper/2312.14190/) (2023)
- [Learning new physics efficiently with nonparametric methods](https://mlsanity.com/paper/2204.02317/) (2022)
- [Choosing Between Methods of Combining p-values](https://mlsanity.com/paper/1707.06897/) (2017)
- [Goodness of fit by Neyman-Pearson testing](https://mlsanity.com/paper/2305.14137/) (2023)
- [Fast kernel methods for Data Quality Monitoring as a goodness-of-fit test](https://mlsanity.com/paper/2303.05413/) (2023)

## Built on by (most cited first, 7 of 7)

- [Comparing Generative Models with the New Physics Learning Machine](https://mlsanity.com/paper/2508.02275/) (2025)
- [Learning to Validate Generative Models: a Goodness-of-Fit Approach](https://mlsanity.com/paper/2511.09118/) (2025)
- [Model-Agnostic Signal Discovery with Machine Learning: Bridging the Gap Between Theory and Practice](https://mlsanity.com/paper/2605.31103/) (2026)
- [AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing](https://mlsanity.com/paper/2510.21935/) (2025)
- [Regularized f-Divergence Kernel Tests](https://mlsanity.com/paper/2601.19755/) (2026)
- [Are We Ready for AI-Driven Discovery? AI Verification Before the Next Fundamental Physics Breakthrough](https://mlsanity.com/paper/2607.10039/) (2026)
- [Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies](https://mlsanity.com/paper/2511.03095/) (2025)

## Abstract (arXiv)

In this work, we address the question of how to enhance signal-agnostic searches by leveraging multiple testing strategies. Specifically, we consider hypothesis tests relying on machine learning, where model selection can introduce a bias towards specific families of new physics signals. We show that it is beneficial to combine different tests, characterised by distinct choices of hyperparameters, and that performances comparable to the best available test are generally achieved while providing a more uniform response to various types of anomalies. Focusing on the New Physics Learning Machine, a methodology to perform a signal-agnostic likelihood-ratio test, we explore a number of approaches to multiple testing, such as combining p-values and aggregating test statistics.