# Kernel Embeddings and the Separation of Measure Phenomenon

arXiv: https://arxiv.org/abs/2505.04613 · HTML: https://mlsanity.com/paper/2505.04613/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 54 references that are in the map.
- First posted on arXiv 7 May 2025.

## Builds on (most cited first, 11 of 11)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [MMD Aggregated Two-Sample Test](https://mlsanity.com/paper/2110.15073/) (2021)
- [Information Theory with Kernel Methods](https://mlsanity.com/paper/2202.08545/) (2022)
- [A new graph-based two-sample test for multivariate and object data](https://mlsanity.com/paper/1307.6294/) (2013)
- [A Permutation-free Kernel Two-Sample Test](https://mlsanity.com/paper/2211.14908/) (2022)
- [Spectral Regularized Kernel Two-Sample Tests](https://mlsanity.com/paper/2212.09201/) (2022)
- [Boosting the Power of Kernel Two-Sample Tests](https://mlsanity.com/paper/2302.10687/) (2023)
- [The Functional Graphical Lasso](https://mlsanity.com/paper/2306.02347/) (2023)
- [The Completion of Covariance Kernels](https://mlsanity.com/paper/2107.07350/) (2021)

## Built on by (most cited first, 2 of 2)

- [Likelihood Ratio Tests by Kernel Gaussian Embedding](https://mlsanity.com/paper/2508.07982/) (2025)
- [PHINN: Persistent Homology Inspired Neural Network for Rare-Event Time Series Generation](https://mlsanity.com/paper/2606.15452/) (2026)

## Abstract (arXiv)

We prove that kernel covariance embeddings lead to information-theoretically perfect separation of distinct continuous probability distributions. In statistical terms, we establish that testing for the equality of two non-atomic (Borel) probability measures on a locally compact uncountable Polish space is equivalent to testing for the singularity between two centered Gaussian measures on a reproducing kernel Hilbert space. The corresponding Gaussians are defined via the notion of kernel covariance embedding of a probability measure, and the Hilbert space is that generated by the embedding kernel. Distinguishing singular Gaussians is structurally simpler from an information-theoretic perspective than non-parametric two-sample testing, particularly in complex or high-dimensional domains. This is because singular Gaussians are supported on essentially separate and affine subspaces. Our proof leverages the classical Feldman-H'ajek dichotomy, and shows that even a small perturbation of a continuous distribution will be maximally magnified through its Gaussian embedding. This ``separation of measure phenomenon'' appears to be a blessing of infinite dimensionality, by means of embedding, with the potential to inform the design of efficient inference tools in considerable generality. The elicitation of this phenomenon also appears to crystallize, in a precise and simple mathematical statement, a core mechanism underpinning the empirical effectiveness of kernel methods.