# Kernel Density Machines

arXiv: https://arxiv.org/abs/2504.21419 · HTML: https://mlsanity.com/paper/2504.21419/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 63 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 16 of 16)

- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Engression: Extrapolation through the Lens of Distributional Regression](https://mlsanity.com/paper/2307.00835/) (2023)
- [Kernel-based Tests for Joint Independence](https://mlsanity.com/paper/1603.00285/) (2016)
- [Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluations](https://mlsanity.com/paper/2207.06503/) (2022)
- [A Rigorous Theory of Conditional Mean Embeddings](https://mlsanity.com/paper/1912.00671/) (2019)
- [Isotonic Distributional Regression](https://mlsanity.com/paper/1909.03725/) (2019)
- [An elementary analysis of ridge regression with random design](https://mlsanity.com/paper/2203.08564/) (2022)
- [Kernel Conditional Density Operators](https://mlsanity.com/paper/1905.11255/) (2019)
- [deepregression: a Flexible Neural Network Framework for Semi-Structured Deep Distributional Regression](https://mlsanity.com/paper/2104.02705/) (2021)
- [On regularized Radon-Nikodym differentiation](https://mlsanity.com/paper/2308.07887/) (2023)
- [A New Parametrization of Correlation Matrices](https://mlsanity.com/paper/2012.02395/) (2020)
- [The Nystr"om method for convex loss functions](https://mlsanity.com/paper/2006.10016/) (2020)
- [Adaptive learning of density ratios in RKHS](https://mlsanity.com/paper/2307.16164/) (2023)
- [Adaptive joint distribution learning](https://mlsanity.com/paper/2110.04829/) (2021)

## Built on by (most cited first, 2 of 2)

- [Kernel-Based Nonparametric Tests For Shape Constraints](https://mlsanity.com/paper/2510.16745/) (2025)
- [Factorizable joint shift revisited](https://mlsanity.com/paper/2601.15036/) (2026)

## Abstract (arXiv)

We introduce kernel density machines (KDM), an agnostic kernel-based framework for learning the Radon-Nikodym derivative (density) between probability measures under minimal assumptions. KDM applies to general measurable spaces and avoids the structural requirements common in classical nonparametric density estimators. We construct a sample estimator and prove its consistency and a functional central limit theorem. To enable scalability, we develop Nystrom-type low-rank approximations and derive optimal error rates, filling a gap in the literature where such guarantees for density learning have been missing. We demonstrate the versatility of KDM through applications to kernel-based two-sample testing and conditional distribution estimation, the latter enjoying dimension-free guarantees beyond those of locally smoothed methods. Experiments on simulated and real data show that KDM is accurate, scalable, and competitive across a range of tasks.