# Nonparametric Estimation of Renyi Divergence and Friends

arXiv: https://arxiv.org/abs/1402.2966 · HTML: https://mlsanity.com/paper/1402.2966/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2014.

## Built on by (most cited first, 8 of 8)

- [Plugin Estimation of Smooth Optimal Transport Maps](https://mlsanity.com/paper/2107.12364/) (2021)
- [Non-Asymptotic Performance Guarantees for Neural Estimation of mathsff-Divergences](https://mlsanity.com/paper/2103.06923/) (2021)
- [The Fundamental Limits of Structure-Agnostic Functional Estimation](https://mlsanity.com/paper/2305.04116/) (2023)
- [The Representation Jensen-Shannon Divergence](https://mlsanity.com/paper/2305.16446/) (2023)
- [When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimality](https://mlsanity.com/paper/2305.16527/) (2023)
- [Auditing Differential Privacy in High Dimensions with the Kernel Quantum R'enyi Divergence](https://mlsanity.com/paper/2205.13941/) (2022)
- [Collaborative likelihood-ratio estimation over graphs](https://mlsanity.com/paper/2205.14461/) (2022)
- [Variational Weighting for Kernel Density Ratios](https://mlsanity.com/paper/2311.03001/) (2023)

## Abstract (arXiv)

We consider nonparametric estimation of L2, Renyi-α and Tsallis-α divergences between continuous distributions. Our approach is to construct estimators for particular integral functionals of two densities and translate them into divergence estimators. For the integral functionals, our estimators are based on corrections of a preliminary plug-in estimator. We show that these estimators achieve the parametric convergence rate of n-1/2 when the densities' smoothness, s, are both at least d/4 where d is the dimension. We also derive minimax lower bounds for this problem which confirm that s > d/4 is necessary to achieve the n-1/2 rate of convergence. We validate our theoretical guarantees with a number of simulations.