mlsanityData to 14 Oct 2026

Multi-modal contrastive learning adapts to intrinsic dimensions of shared latent variables

arXiv 2505.12473 ↗First posted 18 May 2025stat.ML cs.LG math.STcited by 7

Not on the map. Why?

Abstract (arXiv)

Multi-modal contrastive learning as a self-supervised representation learning technique has achieved great success in foundation model training, such as CLIP~citepradford2021learning. In this paper, we study the theoretical properties of the learned representations from multi-modal contrastive learning beyond linear representations and specific data distributions. Our analysis reveals that, enabled by temperature optimization, multi-modal contrastive learning not only maximizes mutual information between modalities but also adapts to intrinsic dimensions of data, which can be much lower than user-specified dimensions for representation vectors. Experiments on both synthetic and real-world datasets demonstrate the ability of contrastive learning to learn low-dimensional and informative representations, bridging theoretical insights and practical performance.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 38 of its 85 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

38 of 85 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  4. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  5. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  7. 2019Contrastive Multiview Codingcited by 1,301
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  9. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  10. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  11. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  12. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
26 more
  1. 2019On Variational Bounds of Mutual Informationcited by 559
  2. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  3. 2022Multimodal Learning with Transformers: A Surveycited by 400
  4. 2020Understanding the Behaviour of Contrastive Losscited by 398
  5. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  6. 2019On Mutual Information Maximization for Representation Learningcited by 316
  7. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  8. 2023Multimodal Deep Learningcited by 258
  9. 2023A Cookbook of Self-Supervised Learningcited by 250
  10. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  11. 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 198
  12. 2020Learning Robust Representations via Multi-View Information Bottleneckcited by 191
  13. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 153
  14. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 83
  15. 2021Self-Supervised Learning with Kernel Dependence Maximizationcited by 77
  16. 2021Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Searchcited by 57
  17. 2015Jointly Modeling Embedding and Translation to Bridge Video and Languagecited by 53
  18. 2023Understanding Multimodal Contrastive Learning and Incorporating Unpaired Datacited by 43
  19. 2015Deep Multimodal Learning for Audio-Visual Speech Recognitioncited by 38
  20. 2023Understanding Transferable Representation Learning and Zero-shot Transfer in CLIPcited by 26
  21. 2023A Theory of Multimodal Learningcited by 25
  22. 2025A Statistical Theory of Contrastive Pre-training and Multimodal Generative AIcited by 13
  23. 2023On the Importance of Contrastive Loss in Multimodal Learningcited by 8
  24. 2023Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkagecited by 6
  25. 2024Multimodal data integration and cross-modal querying via orchestrated approximate message passingcited by 5
  26. 2025A Statistical Theory of Contrastive Learning via Approximate Sufficient Statisticscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Global Minimizers of Sigmoid Contrastive Losscited by 5
  2. 2025IndiSeek learns information-guided disentangled representationscited by 2
  3. 2026Multi-layer Cross-attention is Provably Optimal for Multi-modal In-context Learningcited by 2
  4. 2026Learning Molecular Representations from Cellular Phenotypes with Structure Preservationcited by 0
  5. 2026DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passingcited by 0
  6. 2026FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Datacited by 0
  7. 2026Expressivity In Multimodal Contrastive Learningcited by 0