# The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal

arXiv: https://arxiv.org/abs/1711.08824 · HTML: https://mlsanity.com/paper/1711.08824/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2017.

## Built on by (most cited first, 4 of 4)

- [Behavior From the Void: Unsupervised Active Pre-Training](https://mlsanity.com/paper/2103.04551/) (2021)
- [High-Dimensional Smoothed Entropy Estimation via Dimensionality Reduction](https://mlsanity.com/paper/2305.04712/) (2023)
- [Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Tasks](https://mlsanity.com/paper/2211.06134/) (2022)
- [CIM: Constrained Intrinsic Motivation for Sparse-Reward Continuous Control](https://mlsanity.com/paper/2211.15205/) (2022)

## Abstract (arXiv)

We analyze the Kozachenko--Leonenko (KL) nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance over H"older balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a new minimax lower bound over the H"older ball, we show that the KL estimator is achieving the minimax rates up to logarithmic factors without cognizance of the smoothness parameter s of the H"older ball for s∈ (0,2] and arbitrary dimension d, rendering it the first estimator that provably satisfies this property.