# Minimax Optimal Estimation of KL Divergence for Continuous Distributions

arXiv: https://arxiv.org/abs/2002.11599 · HTML: https://mlsanity.com/paper/2002.11599/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 11 of 11)

- [On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2102.05485/) (2021)
- [Incentives in Private Collaborative Machine Learning](https://mlsanity.com/paper/2404.01676/) (2024)
- [Minimax Optimal Q Learning with Nearest Neighbors](https://mlsanity.com/paper/2308.01490/) (2023)
- [Auditing Differential Privacy in High Dimensions with the Kernel Quantum R'enyi Divergence](https://mlsanity.com/paper/2205.13941/) (2022)
- [Statistical Inference for Generative Model Comparison](https://mlsanity.com/paper/2501.18897/) (2025)
- [SuperMeshing: A New Deep Learning Architecture for Increasing the Mesh Density of Metal Forming Stress Field with Attention Mechanism and Perceptual Features](https://mlsanity.com/paper/2104.09276/) (2021)
- [Embedding Functional Data: Multidimensional Scaling and Manifold Learning](https://mlsanity.com/paper/2208.14540/) (2022)
- [Sample Complexity of Nonparametric Closeness Testing for Continuous Distributions and Its Application to Causal Discovery with Hidden Confounding](https://mlsanity.com/paper/2503.07475/) (2025)
- [Dimension Agnostic Testing of Survey Data Credibility through the Lens of Regression](https://mlsanity.com/paper/2508.20616/) (2025)
- [Implicit Hypothesis Testing and Divergence Preservation in Neural Network Representations](https://mlsanity.com/paper/2601.20477/) (2026)
- [Robust Transfer Learning with Side Information](https://mlsanity.com/paper/2603.07921/) (2026)

## Abstract (arXiv)

Estimating Kullback-Leibler divergence from identical and independently distributed samples is an important problem in various domains. One simple and effective estimator is based on the k nearest neighbor distances between these samples. In this paper, we analyze the convergence rates of the bias and variance of this estimator. Furthermore, we derive a lower bound of the minimax mean square error and show that kNN method is asymptotically rate optimal.