# Multivariate f-Divergence Estimation With Confidence

arXiv: https://arxiv.org/abs/1411.2045 · HTML: https://mlsanity.com/paper/1411.2045/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2014.

## Built on by (most cited first, 7 of 7)

- [On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2102.05485/) (2021)
- [Evaluating State-of-the-Art Classification Models Against Bayes Optimality](https://mlsanity.com/paper/2106.03357/) (2021)
- [The Representation Jensen-Shannon Divergence](https://mlsanity.com/paper/2305.16446/) (2023)
- [Robust Generative Learning with Lipschitz-Regularized α-Divergences Allows Minimal Assumptions on Target Distributions](https://mlsanity.com/paper/2405.13962/) (2024)
- [Detecting Distributional Differences in Labeled Sequence Data with Application to Tropical Cyclone Satellite Imagery](https://mlsanity.com/paper/2202.02253/) (2022)
- [Collaborative likelihood-ratio estimation over graphs](https://mlsanity.com/paper/2205.14461/) (2022)
- [Approximating f-Divergences with Rank Statistics](https://mlsanity.com/paper/2601.22784/) (2026)

## Abstract (arXiv)

The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In particular, even for those estimators whose MSE convergence rates are known, the asymptotic distributions are unknown. We establish the asymptotic normality of a recently proposed ensemble estimator of f-divergence between two distributions from a finite number of samples. This estimator has MSE convergence rate of O(1/T), is simple to implement, and performs well in high dimensions. This theory enables us to perform divergence-based inference tasks such as testing equality of pairs of distributions based on empirical samples. We experimentally validate our theoretical results and, as an illustration, use them to empirically bound the best achievable classification error.