# Advanced Tutorial: Label-Efficient Two-Sample Tests

arXiv: https://arxiv.org/abs/2501.03568 · HTML: https://mlsanity.com/paper/2501.03568/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 36 references that are in the map.
- First posted on arXiv 7 Jan 2025.

## Builds on (most cited first, 6 of 6)

- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [A new graph-based two-sample test for multivariate and object data](https://mlsanity.com/paper/1307.6294/) (2013)
- [Sequential Nonparametric Testing with the Law of the Iterated Logarithm](https://mlsanity.com/paper/1506.03486/) (2015)
- [S2: An Efficient Graph Based Active Learning Algorithm with Application to Nonparametric Classification](https://mlsanity.com/paper/1506.08760/) (2015)
- [A label-efficient two-sample test](https://mlsanity.com/paper/2111.08861/) (2021)
- [Active Sequential Two-Sample Testing](https://mlsanity.com/paper/2301.12616/) (2023)

## Built on by (most cited first, 2 of 2)

- [Statistically Valid Post-Deployment Monitoring Should Be Standard for AI-Based Digital Health](https://mlsanity.com/paper/2506.05701/) (2025)
- [Pay Only for Disagreement: Certified No-Regression Verdicts for Model Updates with Matching Label-Complexity Bounds](https://mlsanity.com/paper/2609.17560/) (2026)

## Abstract (arXiv)

Hypothesis testing is a statistical inference approach used to determine whether data supports a specific hypothesis. An important type is the two-sample test, which evaluates whether two sets of data points are from identical distributions. This test is widely used, such as by clinical researchers comparing treatment effectiveness. This tutorial explores two-sample testing in a context where an analyst has many features from two samples, but determining the sample membership (or labels) of these features is costly. In machine learning, a similar scenario is studied in active learning. This tutorial extends active learning concepts to two-sample testing within this textitlabel-costly setting while maintaining statistical validity and high testing power. Additionally, the tutorial discusses practical applications of these label-efficient two-sample tests.