# Adaptive Kernel Density Estimation with Pre-training

arXiv: https://arxiv.org/abs/2605.13092 · HTML: https://mlsanity.com/paper/2605.13092/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 11 of 11)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings](https://mlsanity.com/paper/1906.02236/) (2019)
- [More efficient approximation of smoothing splines via space-filling basis selection](https://mlsanity.com/paper/2003.10609/) (2020)
- [DiScoFormer: Plug-In Density and Score Estimation with Transformers](https://mlsanity.com/paper/2511.05924/) (2025)
- [Data-driven deep density estimation](https://mlsanity.com/paper/2107.11085/) (2021)
- [Smoothing splines approximation using Hilbert curve basis selection](https://mlsanity.com/paper/2109.11727/) (2021)
- [Adaptive Basis Selection for Exponential Family Smoothing Splines with Application in Joint Modeling of Multiple Sequencing Samples](https://mlsanity.com/paper/1702.01875/) (2017)

## Abstract (arXiv)

Density estimation in high-dimensional settings is an important and challenging statistical problem.Traditional methods based on kernel smoothing are inefficient in high dimensions due to the difficulties in specifying appropriate location-adaptive kernels. In this work, we introduce pre-training, a key idea behind many cutting-edge AI technologies, to the context of non-parametric density estimation. By establishing a pre-trained neural network that can recommend an appropriate location-adaptive kernel for each sample point, efficient density estimation with adaptive kernels is achieved in high dimensions. A wide range of numerical experiments show that this strategy is highly effective for improving density-estimation accuracy, when the target distribution is close to the distribution family for pre-training. When the target distribution is substantially different from the pre-training distribution family, the benefit from the proposed pre-training strategy may be diluted, but can be reactivated by an additional fine-tuning procedure.