# NMINE: Normalized Mutual Information Neural Estimation

arXiv: https://arxiv.org/abs/2607.27710 · HTML: https://mlsanity.com/paper/2607.27710/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Learning Disentangled Representations via Mutual Information Estimation](https://mlsanity.com/paper/1912.03915/) (2019)
- [Efficient Estimation of Mutual Information for Strongly Dependent Variables](https://mlsanity.com/paper/1411.2003/) (2014)
- [Normalized mutual information is a biased measure for classification and community detection](https://mlsanity.com/paper/2307.01282/) (2023)
- [Deep Data Density Estimation through Donsker-Varadhan Representation](https://mlsanity.com/paper/2104.06612/) (2021)
- [Improving Numerical Stability of Normalized Mutual Information Estimator on High Dimensions](https://mlsanity.com/paper/2410.07642/) (2024)

## Abstract (arXiv)

Mutual information is a general measure of statistical dependence that captures both linear and nonlinear relationships between random variables. For continuous and multidimensional variables For continuous multidimensional variables, mutual information must be estimated from samples. Because mutual information is unbounded, its values are not directly comparable across datasets, dimensions, or applications. Normalized mutual information addresses this limitation by converting mutual information into a normalized dependency score. Recent work has demonstrated the practical value of normalized mutual information in applications such as molecular dynamics {arXiv:2405.04980} and interpretable machine learning {arXiv:2409.16768}, but existing estimators remain sensitive to dimensionality and numerical stability {arXiv:2410.07642}. In this paper, we propose a fully neural normalized mutual information estimator for continuous variables. The proposed approach combines a MINE-based neural mutual information estimator {arXiv:1801.04062} with MI-NEE-inspired neural marginal entropy estimators {arXiv:1905.12957}. Mutual information is estimated using the Donsker--Varadhan representation, while marginal entropies are estimated by learning the divergence between each marginal distribution and a uniform reference distribution, from which entropy is recovered. The resulting estimator provides a neural alternative to k-nearest-neighbor-based normalized mutual information estimation {arXiv:2405.04980}. Experiments on Gaussian data from one to eight dimensions show that the proposed estimator improves accuracy over a KSG-based normalized mutual information baseline. These results indicate that neural estimation is a promising direction for normalized dependency measurement in continuous multidimensional settings.