# A Practical Introduction to Kernel Discrepancies: MMD, HSIC & KSD

arXiv: https://arxiv.org/abs/2503.04820 · HTML: https://mlsanity.com/paper/2503.04820/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 57 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 21 of 21)

- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy](https://mlsanity.com/paper/1611.04488/) (2016)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [MMD Aggregated Two-Sample Test](https://mlsanity.com/paper/2110.15073/) (2021)
- [MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting](https://mlsanity.com/paper/2306.08777/) (2023)
- [Large-Scale Kernel Methods for Independence Testing](https://mlsanity.com/paper/1606.07892/) (2016)
- [Efficient Aggregated Kernel Tests using Incomplete U-statistics](https://mlsanity.com/paper/2206.09194/) (2022)
- [KSD Aggregated Goodness-of-fit Test](https://mlsanity.com/paper/2202.00824/) (2022)
- [A Permutation-free Kernel Two-Sample Test](https://mlsanity.com/paper/2211.14908/) (2022)
- [Efficient Conditionally Invariant Representation Learning](https://mlsanity.com/paper/2212.08645/) (2022)
- [Targeted Separation and Convergence with Kernel Discrepancies](https://mlsanity.com/paper/2209.12835/) (2022)
- [Practical Kernel Tests of Conditional Independence](https://mlsanity.com/paper/2402.13196/) (2024)
- [Dimension-agnostic inference using cross U-statistics](https://mlsanity.com/paper/2011.05068/) (2020)
- [A Permutation-Free Kernel Independence Test](https://mlsanity.com/paper/2212.09108/) (2022)
- [Unbiased estimators for the variance of MMD estimators](https://mlsanity.com/paper/1906.02104/) (2019)
- [Differentially Private Permutation Tests](https://mlsanity.com/paper/2310.19043/) (2023)
- [Robust Kernel Hypothesis Testing under Data Corruption](https://mlsanity.com/paper/2405.19912/) (2024)

## Built on by (most cited first, 13 of 13)

- [DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing](https://mlsanity.com/paper/2510.11140/) (2025)
- [A Unified View of Optimal Kernel Hypothesis Testing](https://mlsanity.com/paper/2503.07084/) (2025)
- [Regularized f-Divergence Kernel Tests](https://mlsanity.com/paper/2601.19755/) (2026)
- [Convolutional Maximum Mean Discrepancy for Inference in Noisy Data](https://mlsanity.com/paper/2604.12022/) (2026)
- [DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlation](https://mlsanity.com/paper/2506.11653/) (2025)
- [Incorporating Interventional Independence Improves Robustness against Interventional Distribution Shift](https://mlsanity.com/paper/2507.05412/) (2025)
- [Linear cost mutual information estimation and independence test of similar performance as HSIC](https://mlsanity.com/paper/2508.18338/) (2025)
- [Mixture Proportion Estimation and Weakly-supervised Kernel Test for Conditional Independence](https://mlsanity.com/paper/2604.07191/) (2026)
- [Nonparametric Deconvolution and Denoising using Simulation Based Inference](https://mlsanity.com/paper/2606.21907/) (2026)
- [Aggregation of Statistical Evidence under Exchangeability](https://mlsanity.com/paper/2607.15823/) (2026)
- [Sliced Orlicz-Wasserstein](https://mlsanity.com/paper/2609.32847/) (2026)
- [Unbounded Characteristic and Universal Kernels](https://mlsanity.com/paper/2610.09731/) (2026)
- [On the Optimisation Landscape of Overparameterised Weighted MMD](https://mlsanity.com/paper/2610.19861/) (2026)

## Abstract (arXiv)

This article provides a practical introduction to kernel discrepancies, focusing on the Maximum Mean Discrepancy (MMD), the Hilbert-Schmidt Independence Criterion (HSIC), and the Kernel Stein Discrepancy (KSD). Various estimators for these discrepancies are presented, including the commonly-used V-statistics and U-statistics, as well as several forms of the more computationally-efficient incomplete U-statistics. The importance of the choice of kernel bandwidth is stressed, showing how it affects the behaviour of the discrepancy estimation. Adaptive estimators are introduced, which combine multiple estimators with various kernels, addressing the problem of kernel selection.