mlsanityData to 7 Oct 2026

Deep Variational Information Bottleneck

arXiv 1612.00410 ↗First posted 1 Dec 2016cs.LG cs.IT math.ITcited by 915

Not on the map. Why?

Cited by 21 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a variational approximation to the information bottleneck of Tishby et al. (1999). This variational approach allows us to parameterize the information bottleneck model using a neural network and leverage the reparameterization trick for efficient training. We call this method "Deep Variational Information Bottleneck", or Deep VIB. We show that models trained with the VIB objective outperform those that are trained with other forms of regularization, in terms of generalization performance and robustness to adversarial attack.

Key facts. Cited by 915 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 2; 2020: 3; 2021: 136; 2022: 146; 2023: 155; 2024: 137; 2025: 156; 2026: 179.

915 papers.

Sort:

By area:

  1. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  2. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  3. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  4. 2022Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanismcited by 171
  5. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  6. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 166
  7. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  8. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
  9. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  10. 2022Disentangled Representation Learningcited by 111
  11. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  12. 2021Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approachcited by 105
38 more
  1. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 103
  2. 2023How Does Information Bottleneck Help Deep Learning?cited by 92
  3. 2023Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Modelscited by 90
  4. 2021Make It Move: Controllable Image-to-Video Generation with Text Descriptionscited by 88
  5. 2021Invariant Information Bottleneck for Domain Generalizationcited by 86
  6. 2021Graph Structure Learning with Variational Information Bottleneckcited by 81
  7. 2022An Introduction to Neural Data Compressioncited by 78
  8. 2024InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modelingcited by 76
  9. 2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
  10. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  11. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  12. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
  13. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  14. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  15. 2021Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristicscited by 67
  16. 2022Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Featurescited by 63
  17. 2021Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regressioncited by 58
  18. 2024Parallelized Autoregressive Visual Generationcited by 58
  19. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  20. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Modelcited by 56
  21. 2023Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimizationcited by 56
  22. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 55
  23. 2023Task-specific Fine-tuning via Variational Information Bottleneck for Weakly-supervised Pathology Whole Slide Image Classificationcited by 54
  24. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  25. 2022SdAE: Self-distillated Masked Autoencodercited by 52
  26. 2021Variational Information Bottleneck for Effective Low-Resource Fine-Tuningcited by 51
  27. 2021Lossy Compression for Lossless Predictioncited by 51
  28. 2021Optimal Representations for Covariate Shiftcited by 51
  29. 2021Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identificationcited by 50
  30. 2021Rectangular Flows for Manifold Learningcited by 49
  31. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  32. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  33. 2021Task-Oriented Communication for Multi-Device Cooperative Edge Inferencecited by 47
  34. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  35. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 46
  36. 2021Controllable Guarantees for Fair Outcomes via Contrastive Information Estimationcited by 45
  37. 2021Improving Subgraph Recognition with Variational Graph Information Bottleneckcited by 45
  38. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45

50 most cited of 915 All 915 (JSON)