Minimax Optimal Estimation of KL Divergence for Continuous Distributions
Not on the map. Why?
Abstract (arXiv)
Estimating Kullback-Leibler divergence from identical and independently distributed samples is an important problem in various domains. One simple and effective estimator is based on the k nearest neighbor distances between these samples. In this paper, we analyze the convergence rates of the bias and variance of this estimator. Furthermore, we derive a lower bound of the minimax mean square error and show that kNN method is asymptotically rate optimal.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Feb 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 2; 2022: 2; 2023: 1; 2024: 1; 2025: 3; 2026: 2.
11 papers.
- 2021On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributionscited by 23
- 2024Incentives in Private Collaborative Machine Learningcited by 10
- 2023Minimax Optimal Q Learning with Nearest Neighborscited by 8
- 2022Auditing Differential Privacy in High Dimensions with the Kernel Quantum R'enyi Divergencecited by 3
- 2025Statistical Inference for Generative Model Comparisoncited by 1
- 2021SuperMeshing: A New Deep Learning Architecture for Increasing the Mesh Density of Metal Forming Stress Field with Attention Mechanism and Perceptual Featurescited by 0
- 2022Embedding Functional Data: Multidimensional Scaling and Manifold Learningcited by 0
- 2025Sample Complexity of Nonparametric Closeness Testing for Continuous Distributions and Its Application to Causal Discovery with Hidden Confoundingcited by 0
- 2025Dimension Agnostic Testing of Survey Data Credibility through the Lens of Regressioncited by 0
- 2026Implicit Hypothesis Testing and Divergence Preservation in Neural Network Representationscited by 0
- 2026Robust Transfer Learning with Side Informationcited by 0
- 2026Robust Transfer Learning with Side Information9 Mar 2026
- 2026Implicit Hypothesis Testing and Divergence Preservation in Neural Network Representations28 Jan 2026
- 2025Dimension Agnostic Testing of Survey Data Credibility through the Lens of Regression28 Aug 2025
- 2025Sample Complexity of Nonparametric Closeness Testing for Continuous Distributions and Its Application to Causal Discovery with Hidden Confounding10 Mar 2025
- 2025Statistical Inference for Generative Model Comparison31 Jan 2025
- 2024Incentives in Private Collaborative Machine Learning2 Apr 2024
- 2023Minimax Optimal Q Learning with Nearest Neighbors3 Aug 2023
- 2022Embedding Functional Data: Multidimensional Scaling and Manifold Learning30 Aug 2022
- 2022Auditing Differential Privacy in High Dimensions with the Kernel Quantum R'enyi Divergence27 May 2022
- 2021SuperMeshing: A New Deep Learning Architecture for Increasing the Mesh Density of Metal Forming Stress Field with Attention Mechanism and Perceptual Features12 Mar 2021
- 2021On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions10 Feb 2021