mlsanityData to 14 Oct 2026

Learning Robust Representations via Multi-View Information Bottleneck

arXiv 2002.07017 ↗First posted 17 Feb 2020cs.LG stat.MLcited by 191

Not on the map. Why?

Abstract (arXiv)

The information bottleneck principle provides an information-theoretic method for representation learning, by training an encoder to retain all information which is relevant for predicting the label while minimizing the amount of other, excess information in the representation. The original formulation, however, requires labeled data to identify the superfluous information. In this work, we extend this ability to the multi-view unsupervised setting, where two views of the same underlying entity are provided but the label is unknown. This enables us to identify superfluous information as that not shared by both views. A theoretical analysis leads to the definition of a new multi-view model that produces state-of-the-art results on the Sketchy dataset and label-limited versions of the MIR-Flickr dataset. We also extend our theory to the single-view setting by taking advantage of standard data augmentation techniques, empirically showing better generalization capabilities when compared to common unsupervised approaches for representation learning.

Key facts. Cited by 191 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 22; 2022: 35; 2023: 44; 2024: 37; 2025: 35; 2026: 18.

191 papers.

Sort:

By area:

  1. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 153
  2. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 141
  3. 2022Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)cited by 122
  4. 2023How Does Information Bottleneck Help Deep Learning?cited by 93
  5. 2021SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentationcited by 86
  6. 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
  7. 2023Factorized Contrastive Learning: Going Beyond Multi-view Redundancycited by 75
  8. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  9. 2022Rethinking Minimal Sufficient Representation in Contrastive Learningcited by 69
  10. 2023Identifiability Results for Multimodal Contrastive Learningcited by 61
  11. 2021Distilling Audio-Visual Knowledge by Compositional Contrastive Learningcited by 53
  12. 2022Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCocited by 52
38 more
  1. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  2. 2021Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clusteringcited by 51
  3. 2021Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identificationcited by 50
  4. 2024Training of Physical Neural Networkscited by 49
  5. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  6. 2021Compressive Visual Representationscited by 41
  7. 2023Towards Self-Interpretable Graph-Level Anomaly Detectioncited by 41
  8. 2024Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Predictioncited by 41
  9. 2021A Variational Information Bottleneck Approach to Multi-Omics Data Integrationcited by 40
  10. 2022Is a Caption Worth a Thousand Images? A Controlled Study for Representation Learningcited by 39
  11. 2023Shape-Erased Feature Learning for Visible-Infrared Person Re-Identificationcited by 38
  12. 2022Improving Self-Supervised Learning by Characterizing Idealized Representationscited by 34
  13. 2021DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneckcited by 31
  14. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  15. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 25
  16. 2023An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularizationcited by 25
  17. 2023A Theory of Multimodal Learningcited by 25
  18. 2021Unsupervised Hashing with Contrastive Information Bottleneckcited by 24
  19. 2022Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimationcited by 24
  20. 2022Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representationscited by 21
  21. 2024Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneckcited by 21
  22. 2023CroSSL: Cross-modal Self-Supervised Learning for Time-series through Latent Maskingcited by 19
  23. 2023Selective Visual Representations Improve Convergence and Generalization for Embodied AIcited by 19
  24. 2022MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspectivecited by 17
  25. 2024An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generationcited by 16
  26. 2021Heterogeneous Contrastive Learningcited by 15
  27. 2022Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clusteringcited by 15
  28. 2022What Do We Maximize in Self-Supervised Learning?cited by 15
  29. 2024Fine-Tuning Language Models with Reward Learning on Policycited by 15
  30. 2025Learning Optimal Multimodal Information Bottleneck Representationscited by 15
  31. 2022Abstracting Sketches through Simple Primitivescited by 14
  32. 2023A Novel Approach for Effective Multi-View Clustering with Information-Theoretic Perspectivecited by 14
  33. 2023Dynamic Multimodal Information Bottleneck for Multimodality Classificationcited by 14
  34. 2024Demonstrating and Reducing Shortcuts in Vision-Language Representation Learningcited by 14
  35. 2024An Information Criterion for Controlled Disentanglement of Multimodal Datacited by 14
  36. 2021Leveraging Information Bottleneck for Scientific Document Summarizationcited by 13
  37. 2025Aligning Multimodal Representations through an Information Bottleneckcited by 13
  38. 2021Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learningcited by 12

50 most cited of 191 All 191 (JSON)