mlsanityData to 7 Oct 2026

Poly-View Contrastive Learning

arXiv 2403.05490 ↗First posted 8 Mar 2024cs.LG cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Contrastive learning typically matches pairs of related views among a number of unrelated negative views. Views can be generated (e.g. by augmentations) or be observed. We investigate matching when there are more than two related views which we call poly-view tasks, and derive new representation learning objectives using information maximization and sufficient statistics. We show that with unlimited computation, one should maximize the number of related views, and with a fixed compute budget, it is beneficial to decrease the number of unique samples whilst increasing the number of views of those samples. In particular, poly-view contrastive models trained for 128 epochs with batch size 256 outperform SimCLR trained for 1024 epochs at batch size 4096 on ImageNet1k, challenging the belief that contrastive models require large batch sizes and many training epochs.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 50 of its 72 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

50 of 72 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  12. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
38 more
  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  5. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  7. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  8. 2017Deep Setscited by 1,394
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  10. 2019Contrastive Multiview Codingcited by 1,297
  11. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  12. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  13. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  14. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  15. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  16. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  17. 2020Contrastive Learning with Hard Negative Samplescited by 619
  18. 2019On Variational Bounds of Mutual Informationcited by 550
  19. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  20. 2019On Mutual Information Maximization for Representation Learningcited by 315
  21. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  22. 2023A Cookbook of Self-Supervised Learningcited by 248
  23. 2018An efficient framework for learning sentence representationscited by 197
  24. 2023Stabilizing Transformer Training by Preventing Attention Entropy Collapsecited by 154
  25. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 146
  26. 2021Contrastive Learning with Stronger Augmentationscited by 100
  27. 2020Neural Approximate Sufficient Statistics for Implicit Modelscited by 72
  28. 2022Rethinking Minimal Sufficient Representation in Contrastive Learningcited by 67
  29. 2022The Effects of Regularization and Data Augmentation are Class Dependentcited by 67
  30. 2019Augment your batch: better training with larger batchescited by 37
  31. 2020Multi-label Contrastive Predictive Codingcited by 32
  32. 2021Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Errorcited by 32
  33. 2023An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularizationcited by 25
  34. 2017Contrastive-center loss for deep neural networkscited by 20
  35. 2022A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Momentscited by 14
  36. 2023The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learningcited by 13
  37. 2023Towards a Rigorous Analysis of Mutual Information in Contrastive Learningcited by 2
  38. 2023ScoreCL: Augmentation-Adaptive Contrastive Learning via Score-Matching Functioncited by 1

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 4.

9 papers.

  1. 2024Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalitiescited by 23
  2. 2024Wearable Accelerometer Foundation Models for Health via Knowledge Distillationcited by 15
  3. 2024What Variables Affect Out-of-Distribution Generalization in Pretrained Models?cited by 13
  4. 2024Contrasting Multiple Representations with the Multi-Marginal Matching Gapcited by 8
  5. 2025A Principled Framework for Multi-View Contrastive Learningcited by 4
  6. 2025Learning ECG Representations via Poly-Window Contrastive Learningcited by 1
  7. 2024Mutual Information Multinomial Estimationcited by 0
  8. 2025Divergence-Based Similarity Function for Multi-View Contrastive Learningcited by 0
  9. 2025Task-Level Contrastiveness for Cross-Domain Few-Shot Learningcited by 0