mlsanityData to 7 Oct 2026

Mutual Information Estimation via Normalizing Flows

arXiv 2403.02187 ↗First posted 4 Mar 2024cs.LG cs.IT math.ITcited by 22

Not on the map. Why?

Abstract (arXiv)

We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI. Theoretical guarantees are provided to demonstrate that our approach yields MI estimates for the original data. Experiments with high-dimensional data are conducted to highlight the practical advantages of the proposed method.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 30 of its 66 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

30 of 66 references.

References · 30

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2015Variational Inference with Normalizing Flowscited by 2,161
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  5. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  6. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  7. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  8. 2018MINE: Mutual Information Neural Estimationcited by 680
  9. 2019On Variational Bounds of Mutual Informationcited by 550
  10. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  11. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
  12. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
18 more
  1. 2020Reasoning About Generalization via Conditional Mutual Informationcited by 145
  2. 2019Understanding the Limitations of Variational Mutual Information Estimatorscited by 140
  3. 2020Flows for simultaneous manifold learning and density estimationcited by 117
  4. 2020Telescoping Density-Ratio Estimationcited by 108
  5. 2021Diffusion Normalizing Flowcited by 77
  6. 2023normflows: A PyTorch Package for Normalizing Flowscited by 56
  7. 2017Model-Powered Conditional Independence Testcited by 43
  8. 2023Beyond Normal: On the Evaluation of Mutual Information Estimatorscited by 37
  9. 2020On Information Plane Analyses of Neural Network Classifiers -- A Reviewcited by 35
  10. 2023MINDE: Mutual Information Neural Diffusion Estimationcited by 31
  11. 2016Efficient multivariate entropy estimation via k-nearest neighbour distancescited by 26
  12. 2017Nonparametric independence testing via mutual informationcited by 22
  13. 2023Information Bottleneck Analysis of Deep Neural Networks via Lossy Compressioncited by 19
  14. 2018Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablationcited by 13
  15. 2018Invertible Autoencoder for domain adaptationcited by 10
  16. 2022Diffeomorphic Information Neural Estimationcited by 10
  17. 2023High-Dimensional Smoothed Entropy Estimation via Dimensionality Reductioncited by 7
  18. 2019A New Lower Bound for Kullback-Leibler Divergence Based on Hammersley-Chapman-Robbins Boundcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 12; 2026: 8.

22 papers.

Sort:

By area:

  1. 2025InfoBridge: Mutual Information estimation via Bridge Matchingcited by 7
  2. 2025Learning of Population Dynamics: Inverse Optimization Meets JKO Schemecited by 6
  3. 2024Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMaxcited by 4
  4. 2025Explaining Grokking and Information Bottleneck through Neural Collapse Emergencecited by 4
  5. 2025Partial Information Decomposition via Normalizing Flows in Latent Gaussian Distributionscited by 4
  6. 2025Neural Mutual Information Estimation with Vector Copulascited by 4
  7. 2025FMMI: Flow Matching Mutual Information Estimationcited by 4
  8. 2025INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimatorscited by 2
  9. 2025Accurate Estimation of Mutual Information in High Dimensional Datacited by 2
  10. 2025A Neural Difference-of-Entropies Estimator for Mutual Informationcited by 1
  11. 2025Multimodal Datasets with Controllable Mutual Informationcited by 1
  12. 2025MIST: Mutual Information Estimation Via Supervised Trainingcited by 1
10 more
  1. 2026Discrete Bridges for Mutual Information Estimationcited by 1
  2. 2026Relative Entropy Estimation in Function Space: Theory and Applications to Trajectory Inferencecited by 1
  3. 2026InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimatecited by 1
  4. 2024Mutual Information Multinomial Estimationcited by 0
  5. 2025Nonparametric Estimation of Joint Entropy via Partitioned Sample-Spacingcited by 0
  6. 2026GeoIB: Geometry-Aware Information Bottleneck via Statistical-Manifold Compressioncited by 0
  7. 2026DIPHINE: Diffusion-based Phi-ID Neural Estimatorcited by 0
  8. 2026Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimationcited by 0
  9. 2026Deep Adaptive Bayesian Screeningcited by 0
  10. 2026ALICE: In-context, Zero-shot, Mutual Information Estimationcited by 0