# Direct Density-Derivative Estimation and Its Application in KL-Divergence Approximation

arXiv: https://arxiv.org/abs/1406.7638 · HTML: https://mlsanity.com/paper/1406.7638/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2014.

## Built on by (most cited first, 2 of 2)

- [Discriminative Estimation of Total Variation Distance: A Fidelity Auditor for Generative Data](https://mlsanity.com/paper/2405.15337/) (2024)
- [Improving importance estimation in covariate shift for providing accurate prediction error](https://mlsanity.com/paper/2402.01450/) (2024)

## Abstract (arXiv)

Estimation of density derivatives is a versatile tool in statistical data analysis. A naive approach is to first estimate the density and then compute its derivative. However, such a two-step approach does not work well because a good density estimator does not necessarily mean a good density-derivative estimator. In this paper, we give a direct method to approximate the density derivative without estimating the density itself. Our proposed estimator allows analytic and computationally efficient approximation of multi-dimensional high-order density derivatives, with the ability that all hyper-parameters can be chosen objectively by cross-validation. We further show that the proposed density-derivative estimator is useful in improving the accuracy of non-parametric KL-divergence estimation via metric learning. The practical superiority of the proposed method is experimentally demonstrated in change detection and feature selection.