# MMD Aggregated Two-Sample Test

arXiv: https://arxiv.org/abs/2110.15073 · HTML: https://mlsanity.com/paper/2110.15073/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 73 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy](https://mlsanity.com/paper/1611.04488/) (2016)
- [Exact post-selection inference, with application to the lasso](https://mlsanity.com/paper/1311.6238/) (2013)
- [Optimal Inference After Model Selection](https://mlsanity.com/paper/1410.2597/) (2014)
- [Interpretable Distribution Features with Maximum Testing Power](https://mlsanity.com/paper/1605.06796/) (2016)
- [A Kernel Two-Sample Test for Functional Data](https://mlsanity.com/paper/2008.11095/) (2020)
- [A Witness Two-Sample Test](https://mlsanity.com/paper/2102.05573/) (2021)
- [On the Optimality of Gaussian Kernel Based Nonparametric Tests against Smooth Alternatives](https://mlsanity.com/paper/1909.03302/) (2019)
- [Learning Kernel Tests Without Data Splitting](https://mlsanity.com/paper/2006.02286/) (2020)
- [Post Selection Inference with Incomplete Maximum Mean Discrepancy Estimator](https://mlsanity.com/paper/1802.06226/) (2018)
- [Statistical Depth Meets Machine Learning: Kernel Mean Embeddings and Depth in Functional Data Analysis](https://mlsanity.com/paper/2105.12778/) (2021)

## Built on by (most cited first, 30 of 75)

- [Normalizing flow neural networks by JKO scheme](https://mlsanity.com/paper/2212.14424/) (2022)
- [MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting](https://mlsanity.com/paper/2306.08777/) (2023)
- [DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems](https://mlsanity.com/paper/2402.04467/) (2024)
- [Efficient Aggregated Kernel Tests using Incomplete U-statistics](https://mlsanity.com/paper/2206.09194/) (2022)
- [KSD Aggregated Goodness-of-fit Test](https://mlsanity.com/paper/2202.00824/) (2022)
- [AutoML Two-Sample Test](https://mlsanity.com/paper/2206.08843/) (2022)
- [Spectral Regularized Kernel Two-Sample Tests](https://mlsanity.com/paper/2212.09201/) (2022)
- [Composite Goodness-of-fit Tests with Kernels](https://mlsanity.com/paper/2111.10275/) (2021)
- [Practical Kernel Tests of Conditional Independence](https://mlsanity.com/paper/2402.13196/) (2024)
- [A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science](https://mlsanity.com/paper/2403.12636/) (2024)
- [A Practical Introduction to Kernel Discrepancies: MMD, HSIC & KSD](https://mlsanity.com/paper/2503.04820/) (2025)
- [A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors](https://mlsanity.com/paper/2310.08287/) (2023)
- [Representation Distribution Matching for One-Step Visual Generation](https://mlsanity.com/paper/2607.02375/) (2026)
- [Boosting the Power of Kernel Two-Sample Tests](https://mlsanity.com/paper/2302.10687/) (2023)
- [The Representation Jensen-Shannon Divergence](https://mlsanity.com/paper/2305.16446/) (2023)
- [A Bias-Variance-Covariance Decomposition of Kernel Scores for Generative Models](https://mlsanity.com/paper/2310.05833/) (2023)
- [Variable Selection for Kernel Two-Sample Tests](https://mlsanity.com/paper/2302.07415/) (2023)
- [Robust Kernel Hypothesis Testing under Data Corruption](https://mlsanity.com/paper/2405.19912/) (2024)
- [A Kernel-Based Conditional Two-Sample Test Using Nearest Neighbors (with Applications to Calibration, Regression Curves, and Simulation-Based Inference)](https://mlsanity.com/paper/2407.16550/) (2024)
- [On the Robustness of Kernel Goodness-of-Fit Tests](https://mlsanity.com/paper/2408.05854/) (2024)
- [Spectral Regularized Kernel Goodness-of-Fit Tests](https://mlsanity.com/paper/2308.04561/) (2023)
- [A Unified Data Representation Learning for Non-parametric Two-sample Testing](https://mlsanity.com/paper/2412.00613/) (2024)
- [A Scalable Nystrom-Based Kernel Two-Sample Test with Permutations](https://mlsanity.com/paper/2502.13570/) (2025)
- [DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing](https://mlsanity.com/paper/2510.11140/) (2025)
- [Variable Selection in Maximum Mean Discrepancy for Interpretable Distribution Comparison](https://mlsanity.com/paper/2311.01537/) (2023)
- [A Unified View of Optimal Kernel Hypothesis Testing](https://mlsanity.com/paper/2503.07084/) (2025)
- [Sequentially Auditing Differential Privacy](https://mlsanity.com/paper/2509.07055/) (2025)
- [Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy](https://mlsanity.com/paper/2304.14762/) (2023)
- [Statistical and Computational Guarantees of Kernel Max-Sliced Wasserstein Distances](https://mlsanity.com/paper/2405.15441/) (2024)
- [General Frameworks for Conditional Two-Sample Testing](https://mlsanity.com/paper/2410.16636/) (2024)

## Abstract (arXiv)

We propose two novel nonparametric two-sample kernel tests based on the Maximum Mean Discrepancy (MMD). First, for a fixed kernel, we construct an MMD test using either permutations or a wild bootstrap, two popular numerical procedures to determine the test threshold. We prove that this test controls the probability of type I error non-asymptotically. Hence, it can be used reliably even in settings with small sample sizes as it remains well-calibrated, which differs from previous MMD tests which only guarantee correct test level asymptotically. When the difference in densities lies in a Sobolev ball, we prove minimax optimality of our MMD test with a specific kernel depending on the smoothness parameter of the Sobolev ball. In practice, this parameter is unknown and, hence, the optimal MMD test with this particular kernel cannot be used. To overcome this issue, we construct an aggregated test, called MMDAgg, which is adaptive to the smoothness parameter. The test power is maximised over the collection of kernels used, without requiring held-out data for kernel selection (which results in a loss of test power), or arbitrary kernel choices such as the median heuristic. We prove that MMDAgg still controls the level non-asymptotically, and achieves the minimax rate over Sobolev balls, up to an iterated logarithmic term. Our guarantees are not restricted to a specific type of kernel, but hold for any product of one-dimensional translation invariant characteristic kernels. We provide a user-friendly parameter-free implementation of MMDAgg using an adaptive collection of bandwidths. We demonstrate that MMDAgg significantly outperforms alternative state-of-the-art MMD-based two-sample tests on synthetic data satisfying the Sobolev smoothness assumption, and that, on real-world image data, MMDAgg closely matches the power of tests leveraging the use of models such as neural networks.