# Kernel Density Estimators in Large Dimensions

arXiv: https://arxiv.org/abs/2408.05807 · HTML: https://mlsanity.com/paper/2408.05807/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 11 Aug 2024.

## Builds on (most cited first, 9 of 9)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Lumiere: A Space-Time Diffusion Model for Video Generation](https://mlsanity.com/paper/2401.12945/) (2024)
- [Dynamical Regimes of Diffusion Models](https://mlsanity.com/paper/2402.18491/) (2024)

## Built on by (most cited first, 5 of 5)

- [Theory of Speciation Transitions in Diffusion Models with General Class Structure](https://mlsanity.com/paper/2602.04404/) (2026)
- [Local Coverage Governs Memorization in Diffusion Models](https://mlsanity.com/paper/2606.14390/) (2026)
- [Geometric Regularity in Deterministic Sampling Dynamics of Diffusion-based Generative Models](https://mlsanity.com/paper/2506.10177/) (2025)
- [Biased Generalization in Diffusion Models](https://mlsanity.com/paper/2603.03469/) (2026)
- [Breaking the Curse with BAND: Nonparametric Distribution Estimation in High Dimensions](https://mlsanity.com/paper/2607.26955/) (2026)

## Abstract (arXiv)

This paper studies Kernel Density Estimation for a high-dimensional distribution ρ(x). Traditional approaches have focused on the limit of large number of data points n and fixed dimension d. We analyze instead the regime where both the number n of data points yi and their dimensionality d grow with a fixed ratio α=(log n)/d. Our study reveals three distinct statistical regimes for the kernel-based estimate of the density hat ρh^mathcal D(x)=frac1n hdsumi=1n Kleft(fracx-yihright), depending on the bandwidth h: a classical regime for large bandwidth where the Central Limit Theorem (CLT) holds, which is akin to the one found in traditional approaches. Below a certain value of the bandwidth, hCLT(α), we find that the CLT breaks down. The statistics of hatρh^mathcal D(x) for a fixed x drawn from ρ(x) is given by a heavy-tailed distribution (an alpha-stable distribution). In particular below a value hG(α), we find that hatρh^mathcal D(x) is governed by extreme value statistics: only a few points in the database matter and give the dominant contribution to the density estimator. We provide a detailed analysis for high-dimensional multivariate Gaussian data. We show that the optimal bandwidth threshold based on Kullback-Leibler divergence lies in the new statistical regime identified in this paper. As known by practitioners, when decreasing the bandwidth a Kernel-estimated estimated changes from a smooth curve to a collections of peaks centred on the data points. Our findings reveal that this general phenomenon is related to sharp transitions between phases characterized by different statistical properties, and offer new insights for Kernel density estimation in high-dimensional settings.