Data to 7 Oct 2026

Learning Representations by Maximizing Mutual Information Across Views

arXiv 1906.00910 ↗First posted 3 Jun 2019cs.LG stat.MLcited by 733

Not on the map. Why?

Abstract (arXiv)

We propose an approach to self-supervised representation learning based on maximizing mutual information between features extracted from multiple views of a shared context. For example, one could produce multiple views of a local spatio-temporal context by observing it from different locations (e.g., camera positions within a scene), and via different modalities (e.g., tactile, auditory, or visual). Or, an ImageNet image could provide a context from which one produces multiple views by repeatedly applying data augmentation. Maximizing mutual information between features extracted from these views requires capturing information about high-level factors whose influence spans multiple views -- e.g., presence of certain objects or occurrence of certain events. Following our proposed approach, we develop a model which learns image representations that significantly outperform prior methods on the tasks we consider. Most notably, using self-supervised learning, our model learns representations which achieve 68.1% accuracy on ImageNet using standard linear evaluation. This beats prior results by over 12% and concurrent results by 7%. When we extend our model to use mixture-based representations, segmentation behaviour emerges as a natural side-effect. Our code is available online: https://github.com/Philip-Bachman/amdim-public.

Key facts. Cited by 733 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 221; 2022: 182; 2023: 153; 2024: 94; 2025: 57; 2026: 26.

733 papers.

Sort:

By area:

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
2021Transformers in Vision: A Surveycited by 1,009
2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
2021Vector-quantized Image Modeling with Improved VQGANcited by 656
2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262

38 more

2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
2023A Cookbook of Self-Supervised Learningcited by 248
2021Contrastive Learning Inverts the Data Generating Processcited by 220
2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 218
2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
2021Towards Causal Representation Learningcited by 207
2021PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformerscited by 204
2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 193
2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 167
2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 167
2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 162
2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
2021Poisoning and Backdooring Contrastive Learningcited by 149
2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
2022Contrastive Masked Autoencoders are Stronger Vision Learnerscited by 133
2021Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learningcited by 130
2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
2021How to represent part-whole hierarchies in a neural networkcited by 123
2021Multimodal Contrastive Training for Visual Representation Learningcited by 122
2021Neighborhood Contrastive Learning for Novel Class Discoverycited by 119
2021An Empirical Study of Graph Contrastive Learningcited by 113
2021Broaden Your Views for Self-Supervised Video Learningcited by 108
2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
2021Contrastive Learning with Stronger Augmentationscited by 100
2021SiT: Self-supervised vIsion Transformercited by 99
2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99
2021Neural Transformation Learning for Deep Anomaly Detection Beyond Imagescited by 98
2021Large-scale Unsupervised Semantic Segmentationcited by 98
2021Region Similarity Representation Learningcited by 96
2022Understanding Contrastive Learning Requires Incorporating Inductive Biasescited by 96
2021Learning to Predict Visual Attributes in the Wildcited by 95

50 most cited of 733 All 733 (JSON)