mlsanityData to 7 Oct 2026

MINE: Mutual Information Neural Estimation

arXiv 1801.04062 ↗First posted 12 Jan 2018cs.LG stat.MLcited by 680

Not on the map. Why?

Abstract (arXiv)

We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size, trainable through back-prop, and strongly consistent. We present a handful of applications on which MINE can be used to minimize or maximize mutual information. We apply MINE to improve adversarially trained generative models. We also use MINE to implement Information Bottleneck, applying it to supervised classification; our results demonstrate substantial improvement in flexibility and performance in these settings.

Key facts. Cited by 680 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 125; 2022: 122; 2023: 116; 2024: 108; 2025: 109; 2026: 99.

680 papers.

Sort:

By area:

  1. 2021How to Train Your Energy-Based Modelscited by 271
  2. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  3. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  4. 2023A Cookbook of Self-Supervised Learningcited by 248
  5. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 184
  6. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  7. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 162
  8. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  9. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  10. 2021Decoupled Contrastive Learningcited by 141
  11. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 138
  12. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
38 more
  1. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  2. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
  3. 2023Modern Bayesian Experimental Designcited by 121
  4. 2023What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalizationcited by 112
  5. 2023Unpaired Image-to-Image Translation via Neural Schr"odinger Bridgecited by 96
  6. 2021Learning Intents behind Interactions with Knowledge Graph for Recommendationcited by 94
  7. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
  8. 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
  9. 2021Graph Structure Learning with Variational Information Bottleneckcited by 81
  10. 2022A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directionscited by 73
  11. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  12. 2022Rethinking Minimal Sufficient Representation in Contrastive Learningcited by 67
  13. 2024AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encodingcited by 67
  14. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
  15. 2022Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Featurescited by 63
  16. 2021Density Ratio Estimation via Infinitesimal Classificationcited by 59
  17. 2021Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identificationcited by 50
  18. 2022Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classificationcited by 50
  19. 2022Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systemscited by 50
  20. 2022Understanding Masked Image Modeling via Learning Occlusion Invariant Featurecited by 50
  21. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  22. 2024Autoformulation of Mathematical Optimization Models Using LLMscited by 48
  23. 2023Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatmentcited by 47
  24. 2023Domain Generalization for Medical Image Analysis: A Reviewcited by 47
  25. 2024Machine Unlearning for Image-to-Image Generative Modelscited by 47
  26. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  27. 2021Learning Disentangled Representations in the Imaging Domaincited by 46
  28. 2021Controllable Guarantees for Fair Outcomes via Contrastive Information Estimationcited by 45
  29. 2021Improving Subgraph Recognition with Variational Graph Information Bottleneckcited by 45
  30. 2021Frustratingly Easy Transferability Estimationcited by 44
  31. 2022Discrete Contrastive Diffusion for Cross-Modal Music and Image Generationcited by 44
  32. 2021Graph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Datacited by 42
  33. 2023Does Invariant Graph Learning via Environment Augmentation Learn Invariance?cited by 42
  34. 2023Backdoor Defense via Deconfounded Representation Learningcited by 38
  35. 2023Beyond Normal: On the Evaluation of Mutual Information Estimatorscited by 37
  36. 2023Interpretable Diffusion via Information Decompositioncited by 37
  37. 2023Multimodal Variational Auto-encoder based Audio-Visual Segmentationcited by 37
  38. 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36

50 most cited of 680 All 680 (JSON)