# Nonparanormal Information Estimation

arXiv: https://arxiv.org/abs/1702.07803 · HTML: https://mlsanity.com/paper/1702.07803/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2017.

## Built on by (most cited first, 4 of 4)

- [Understanding Neural Networks with Logarithm Determinant Entropy Estimator](https://mlsanity.com/paper/2105.03705/) (2021)
- [Reinforcement Learning with Exogenous States and Rewards](https://mlsanity.com/paper/2303.12957/) (2023)
- [Neural Mutual Information Estimation with Vector Copulas](https://mlsanity.com/paper/2510.20968/) (2025)
- [Neuronal Correlation: a Central Concept in Neural Network](https://mlsanity.com/paper/2201.09069/) (2022)

## Abstract (arXiv)

We study the problem of using i.i.d. samples from an unknown multivariate probability distribution p to estimate the mutual information of p. This problem has recently received attention in two settings: (1) where p is assumed to be Gaussian and (2) where p is assumed only to lie in a large nonparametric smoothness class. Estimators proposed for the Gaussian case converge in high dimensions when the Gaussian assumption holds, but are brittle, failing dramatically when p is not Gaussian. Estimators proposed for the nonparametric case fail to converge with realistic sample sizes except in very low dimensions. As a result, there is a lack of robust mutual information estimators for many realistic data. To address this, we propose estimators for mutual information when p is assumed to be a nonparanormal (a.k.a., Gaussian copula) model, a semiparametric compromise between Gaussian and nonparametric extremes. Using theoretical bounds and experiments, we show these estimators strike a practical balance between robustness and scaling with dimensionality.