mlsanityData to 14 Oct 2026

A Neural Difference-of-Entropies Estimator for Mutual Information

arXiv 2502.13085 ↗First posted 18 Feb 2025stat.ML cs.IT cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Estimating Mutual Information (MI), a key measure of dependence of random quantities without specific modelling assumptions, is a challenging problem in high dimensions. We propose a novel mutual information estimator based on parametrizing conditional densities using normalizing flows, a deep generative model that has gained popularity in recent years. This estimator leverages a block autoregressive structure to achieve improved bias-variance trade-offs on standard benchmark tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

20 of 36 references.

References · 20

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2016Density estimation using Real NVPcited by 2,042
  3. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  4. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  5. 2019On Variational Bounds of Mutual Informationcited by 559
  6. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  7. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  8. 2019Understanding the Limitations of Variational Mutual Information Estimatorscited by 140
  9. 2015A Review of Feature Selection Methods Based on Mutual Informationcited by 67
  10. 2019Block Neural Autoregressive Flowcited by 62
  11. 2019CCMI : Classifier based Conditional Mutual Information Estimationcited by 37
  12. 2023Beyond Normal: On the Evaluation of Mutual Information Estimatorscited by 37
8 more
  1. 2023Interpretable Diffusion via Information Decompositioncited by 37
  2. 2023MINDE: Mutual Information Neural Diffusion Estimationcited by 31
  3. 2024Mutual Information Estimation via Normalizing Flowscited by 22
  4. 2016Demystifying Fixed k-Nearest Neighbor Information Estimatorscited by 14
  5. 2022Diffeomorphic Information Neural Estimationcited by 10
  6. 2024A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasetscited by 10
  7. 2020DEMI: Discriminative Estimator of Mutual Informationcited by 5
  8. 2024Numerical Approximation Capacity of Neural Networks with Bounded Parameters: Do Limits Exist, and How Can They Be Measured?cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimationcited by 0