# Refereeing the Referees: Evaluating Two-Sample Tests for Validating Generators in Precision Sciences

arXiv: https://arxiv.org/abs/2409.16336 · HTML: https://mlsanity.com/paper/2409.16336/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 10 of 10)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests](https://mlsanity.com/paper/1509.02237/) (2015)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Evaluating generative models in high energy physics](https://mlsanity.com/paper/2211.10295/) (2022)
- [Learning new physics efficiently with nonparametric methods](https://mlsanity.com/paper/2204.02317/) (2022)
- [Goodness of fit by Neyman-Pearson testing](https://mlsanity.com/paper/2305.14137/) (2023)
- [Testing the boundaries: Normalizing Flows for higher dimensional data sets](https://mlsanity.com/paper/2202.09188/) (2022)

## Built on by (most cited first, 9 of 9)

- [A Scalable Nystrom-Based Kernel Two-Sample Test with Permutations](https://mlsanity.com/paper/2502.13570/) (2025)
- [Comparing Generative Models with the New Physics Learning Machine](https://mlsanity.com/paper/2508.02275/) (2025)
- [Forecasting Generative Amplification](https://mlsanity.com/paper/2509.08048/) (2025)
- [Model-Agnostic Signal Discovery with Machine Learning: Bridging the Gap Between Theory and Practice](https://mlsanity.com/paper/2605.31103/) (2026)
- [Generative Models and Statistical Validation](https://mlsanity.com/paper/2605.30453/) (2026)
- [Minimax-Optimal Two-Sample Test with Sliced Wasserstein](https://mlsanity.com/paper/2510.27498/) (2025)
- [Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies](https://mlsanity.com/paper/2511.03095/) (2025)
- [Calorimeter Shower Superresolution with Conditional Normalizing Flows: Implementation and Statistical Evaluation](https://mlsanity.com/paper/2603.26813/) (2026)
- [Learning Standard Model structure from LHC data with Riemannian flow matching](https://mlsanity.com/paper/2607.16144/) (2026)

## Abstract (arXiv)

We propose a robust methodology to evaluate the performance and computational efficiency of non-parametric two-sample tests, specifically designed for high-dimensional generative models in scientific applications such as in particle physics. The study focuses on tests built from univariate integral probability measures: the sliced Wasserstein distance and the mean of the Kolmogorov-Smirnov statistics, already discussed in the literature, and the novel sliced Kolmogorov-Smirnov statistic. These metrics can be evaluated in parallel, allowing for fast and reliable estimates of their distribution under the null hypothesis. We also compare these metrics with the recently proposed unbiased Fr'echet Gaussian Distance and the unbiased quadratic Maximum Mean Discrepancy, computed with a quartic polynomial kernel. We evaluate the proposed tests on various distributions, focusing on their sensitivity to deformations parameterized by a single parameter ε. Our experiments include correlated Gaussians and mixtures of Gaussians in 5, 20, and 100 dimensions, and a particle physics dataset of gluon jets from the JetNet dataset, considering both jet- and particle-level features. Our results demonstrate that one-dimensional-based tests provide a level of sensitivity comparable to other multivariate metrics, but with significantly lower computational cost, making them ideal for evaluating generative models in high-dimensional settings. This methodology offers an efficient, standardized tool for model comparison and can serve as a benchmark for more advanced tests, including machine-learning-based approaches.