# Revisiting Classifier Two-Sample Tests

arXiv: https://arxiv.org/abs/1610.06545 · HTML: https://mlsanity.com/paper/1610.06545/

## Key facts

- Cited by 268 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2016.

## Built on by (most cited first, 30 of 268)

- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2401.04092/) (2024)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Flow Matching for Scalable Simulation-Based Inference](https://mlsanity.com/paper/2305.17161/) (2023)
- [Ensembling Off-the-shelf Models for GAN Training](https://mlsanity.com/paper/2112.09130/) (2021)
- [Membership Inference Attacks against Synthetic Data through Overfitting Detection](https://mlsanity.com/paper/2302.12580/) (2023)
- [SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation](https://mlsanity.com/paper/2303.12236/) (2023)
- [On Memorization in Probabilistic Deep Generative Models](https://mlsanity.com/paper/2106.03216/) (2021)
- [On Aliased Resizing and Surprising Subtleties in GAN Evaluation](https://mlsanity.com/paper/2104.11222/) (2021)
- [Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models](https://mlsanity.com/paper/2210.04872/) (2022)
- [CaloFlow: Fast and Accurate Generation of Calorimeter Showers with Normalizing Flows](https://mlsanity.com/paper/2106.05285/) (2021)
- [Robust Neural Posterior Estimation and Statistical Model Criticism](https://mlsanity.com/paper/2210.06564/) (2022)
- [Variational methods for simulation-based inference](https://mlsanity.com/paper/2203.04176/) (2022)
- [Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy](https://mlsanity.com/paper/2405.14800/) (2024)
- [Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics](https://mlsanity.com/paper/2405.14806/) (2024)
- [Autoregressive 3D Shape Generation via Canonical Mapping](https://mlsanity.com/paper/2204.01955/) (2022)
- [MAUVE Scores for Generative Models: Theory and Practice](https://mlsanity.com/paper/2212.14578/) (2022)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting](https://mlsanity.com/paper/2306.08777/) (2023)
- [Simulation-Based Inference: A Practical Guide](https://mlsanity.com/paper/2508.12939/) (2025)
- [Evaluating generative models in high energy physics](https://mlsanity.com/paper/2211.10295/) (2022)
- [Truncated Marginal Neural Ratio Estimation](https://mlsanity.com/paper/2107.01214/) (2021)
- [HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles](https://mlsanity.com/paper/2312.11666/) (2023)
- [A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful](https://mlsanity.com/paper/2110.06581/) (2021)
- [SPAGHETTI: Editing Implicit Shapes Through Part Aware Generation](https://mlsanity.com/paper/2201.13168/) (2022)
- [Free-form Flows: Make Any Architecture a Normalizing Flow](https://mlsanity.com/paper/2310.16624/) (2023)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [A Witness Two-Sample Test](https://mlsanity.com/paper/2102.05573/) (2021)

## Abstract (arXiv)

The goal of two-sample tests is to assess whether two samples, SP ~ Pn and SQ ~ Qm, are drawn from the same distribution. Perhaps intriguingly, one relatively unexplored method to build two-sample tests is the use of binary classifiers. In particular, construct a dataset by pairing the n examples in SP with a positive label, and by pairing the m examples in SQ with a negative label. If the null hypothesis "P = Q" is true, then the classification accuracy of a binary classifier on a held-out subset of this dataset should remain near chance-level. As we will show, such Classifier Two-Sample Tests (C2ST) learn a suitable representation of the data on the fly, return test statistics in interpretable units, have a simple null distribution, and their predictive uncertainty allow to interpret where P and Q differ. The goal of this paper is to establish the properties, performance, and uses of C2ST. First, we analyze their main theoretical properties. Second, we compare their performance against a variety of state-of-the-art alternatives. Third, we propose their use to evaluate the sample quality of generative models with intractable likelihoods, such as Generative Adversarial Networks (GANs). Fourth, we showcase the novel application of GANs together with C2ST for causal discovery.