# Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data

arXiv: https://arxiv.org/abs/2106.07636 · HTML: https://mlsanity.com/paper/2106.07636/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 57 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning](https://mlsanity.com/paper/1805.10002/) (2018)
- [On Wasserstein Two Sample Testing and Related Families of Nonparametric Tests](https://mlsanity.com/paper/1509.02237/) (2015)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy](https://mlsanity.com/paper/1611.04488/) (2016)
- [Interpretable Distribution Features with Maximum Testing Power](https://mlsanity.com/paper/1605.06796/) (2016)
- [Fast Two-Sample Testing with Analytic Representations of Probability Measures](https://mlsanity.com/paper/1506.04725/) (2015)
- [Open Set Domain Adaptation: Theoretical Bound and Algorithm](https://mlsanity.com/paper/1907.08375/) (2019)
- [Classification accuracy as a proxy for two sample testing](https://mlsanity.com/paper/1602.02210/) (2016)
- [A Linear-Time Kernel Goodness-of-Fit Test](https://mlsanity.com/paper/1705.07673/) (2017)
- [Attribute Propagation Network for Graph Zero-shot Learning](https://mlsanity.com/paper/2009.11816/) (2020)
- [Robust Hypothesis Testing Using Wasserstein Uncertainty Sets](https://mlsanity.com/paper/1805.10611/) (2018)
- [Two-sample Testing Using Deep Learning](https://mlsanity.com/paper/1910.06239/) (2019)
- [A new graph-based two-sample test for multivariate and object data](https://mlsanity.com/paper/1307.6294/) (2013)
- [CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.10464/) (2020)
- [Learning Kernel Tests Without Data Splitting](https://mlsanity.com/paper/2006.02286/) (2020)
- [Isometric Propagation Network for Generalized Zero-shot Learning](https://mlsanity.com/paper/2102.02038/) (2021)
- [How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?](https://mlsanity.com/paper/2101.01104/) (2020)
- [Neural Tangent Kernel Maximum Mean Discrepancy](https://mlsanity.com/paper/2106.03227/) (2021)
- [Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing](https://mlsanity.com/paper/1508.00655/) (2015)
- [Semantic-Transferable Weakly-Supervised Endoscopic Lesions Segmentation](https://mlsanity.com/paper/1908.07669/) (2019)
- [Sequential Domain Adaptation by Synthesizing Distributionally Robust Experts](https://mlsanity.com/paper/2106.00322/) (2021)
- [Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings](https://mlsanity.com/paper/1906.02236/) (2019)
- [Two-Sample Tests for Large Random Graphs Using Network Statistics](https://mlsanity.com/paper/1705.06168/) (2017)

## Built on by (most cited first, 18 of 18)

- [MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting](https://mlsanity.com/paper/2306.08777/) (2023)
- [Bilateral Dependency Optimization: Defending Against Model-inversion Attacks](https://mlsanity.com/paper/2206.05483/) (2022)
- [Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction](https://mlsanity.com/paper/2205.02708/) (2022)
- [AutoML Two-Sample Test](https://mlsanity.com/paper/2206.08843/) (2022)
- [Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancy](https://mlsanity.com/paper/2402.16041/) (2024)
- [A Unified Data Representation Learning for Non-parametric Two-sample Testing](https://mlsanity.com/paper/2412.00613/) (2024)
- [DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing](https://mlsanity.com/paper/2510.11140/) (2025)
- [Diversity-enhancing Generative Network for Few-shot Hypothesis Adaptation](https://mlsanity.com/paper/2307.05948/) (2023)
- [Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction](https://mlsanity.com/paper/2506.04650/) (2025)
- [AgraSSt: Approximate Graph Stein Statistics for Interpretable Assessment of Implicit Graph Generators](https://mlsanity.com/paper/2203.03673/) (2022)
- [One Stone, Two Birds: Enhancing Adversarial Defense Through the Lens of Distributional Discrepancy](https://mlsanity.com/paper/2503.02169/) (2025)
- [Adversarial Attack and Defense for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2202.03077/) (2022)
- [MLHOps: Machine Learning for Healthcare Operations](https://mlsanity.com/paper/2305.02474/) (2023)
- [Learning Representations for Independence Testing](https://mlsanity.com/paper/2409.06890/) (2024)
- [Multi-class Classification with Fuzzy-feature Observations: Theory and Algorithms](https://mlsanity.com/paper/2206.04311/) (2022)
- [Binary Hypothesis Testing for Softmax Models and Leverage Score Models](https://mlsanity.com/paper/2405.06003/) (2024)
- [Anchor-based Maximum Discrepancy for Relative Similarity Testing](https://mlsanity.com/paper/2510.10477/) (2025)
- [RFFNet: Large-Scale Interpretable Kernel Methods via Random Fourier Features](https://mlsanity.com/paper/2211.06410/) (2022)

## Abstract (arXiv)

Modern kernel-based two-sample tests have shown great success in distinguishing complex, high-dimensional distributions with appropriate learned kernels. Previous work has demonstrated that this kernel learning procedure succeeds, assuming a considerable number of observed samples from each distribution. In realistic scenarios with very limited numbers of data samples, however, it can be challenging to identify a kernel powerful enough to distinguish complex distributions. We address this issue by introducing the problem of meta two-sample testing (M2ST), which aims to exploit (abundant) auxiliary data on related tasks to find an algorithm that can quickly identify a powerful test on new target tasks. We propose two specific algorithms for this task: a generic scheme which improves over baselines and a more tailored approach which performs even better. We provide both theoretical justification and empirical evidence that our proposed meta-testing schemes out-perform learning kernel-based tests directly from scarce observations, and identify when such schemes will be successful.