mlsanityData to 7 Oct 2026

Debiased Contrastive Learning

arXiv 2007.00224 ↗First posted 1 Jul 2020cs.LG stat.MLcited by 354

Not on the map. Why?

Abstract (arXiv)

A prominent technique for self-supervised representation learning has been to contrast semantically similar and dissimilar pairs of samples. Without access to labels, dissimilar (negative) points are typically taken to be randomly sampled datapoints, implicitly accepting that these points may, in reality, actually have the same label. Perhaps unsurprisingly, we observe that sampling negative examples from truly different labels improves performance, in a synthetic setting where labels are available. Motivated by this observation, we develop a debiased contrastive objective that corrects for the sampling of same-label datapoints, even without knowledge of the true labels. Empirically, the proposed objective consistently outperforms the state-of-the-art for representation learning in vision, language, and reinforcement learning benchmarks. Theoretically, we establish generalization bounds for the downstream classification task.

Key facts. Cited by 354 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 76; 2022: 91; 2023: 84; 2024: 33; 2025: 49; 2026: 21.

354 papers.

Sort:

By area:

  1. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  2. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  3. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  4. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  5. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  6. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  7. 2021FSCE: Few-Shot Object Detection via Contrastive Proposal Encodingcited by 168
  8. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  9. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  10. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 121
  11. 2021Can contrastive learning avoid shortcut solutions?cited by 114
  12. 2021An Empirical Study of Graph Contrastive Learningcited by 113
38 more
  1. 2021Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentationcited by 101
  2. 2021Large-scale Unsupervised Semantic Segmentationcited by 98
  3. 2023Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Trainingcited by 91
  4. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  5. 2021Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrastcited by 85
  6. 2021ReSSL: Relational Self-Supervised Learning with Weak Augmentationcited by 83
  7. 2021Weakly Supervised Contrastive Learningcited by 81
  8. 2023Open-Vocabulary Point-Cloud Object Detection without 3D Annotationcited by 81
  9. 2022Rank-N-Contrast: Learning Continuous Representations for Regressioncited by 74
  10. 2021CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representationscited by 72
  11. 2021ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learningcited by 72
  12. 2022Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?cited by 67
  13. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62
  14. 2022Less Data, More Knowledge: Building Next Generation Semantic Communication Networkscited by 61
  15. 2022Hard Sample Aware Network for Contrastive Deep Graph Clusteringcited by 61
  16. 2021Learning Representation for Clustering via Prototype Scattering and Positive Samplingcited by 58
  17. 2023Self-Supervised Multimodal Learning: A Surveycited by 58
  18. 2021Rebooting ACGAN: Auxiliary Classifier GANs with Stable Trainingcited by 57
  19. 2021Investigating the Role of Negatives in Contrastive Representation Learningcited by 54
  20. 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53
  21. 2022Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCocited by 52
  22. 2021Contrastive Learning with Continuous Proxy Meta-Data for 3D MRI Classificationcited by 49
  23. 2022Self-Supervised Vision Transformers Learn Visual Concepts in Histopathologycited by 49
  24. 2021Incremental False Negative Detection for Contrastive Learningcited by 48
  25. 2021Contrastive Learning for Neural Topic Modelcited by 48
  26. 2022ProMix: Combating Label Noise via Maximizing Clean Sample Utilitycited by 48
  27. 2021Self-Supervised Learning Disentangled Group Representation as Featurecited by 47
  28. 2022Robust Contrastive Learning against Noisy Viewscited by 45
  29. 2021Improving Contrastive Learning by Visualizing Feature Transformationcited by 44
  30. 2021Contrastive Multimodal Fusion with TupleInfoNCEcited by 43
  31. 2021Robust Contrastive Learning Using Negative Samples with Diminished Semanticscited by 43
  32. 2022HCSC: Hierarchical Contrastive Selective Codingcited by 43
  33. 2022Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structurecited by 43
  34. 2023TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoderscited by 42
  35. 2024The Double-Ellipsoid Geometry of CLIPcited by 42
  36. 2021Solving Inefficiency of Self-supervised Representation Learningcited by 41
  37. 2022Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performancecited by 40
  38. 2023Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Trainingcited by 39

50 most cited of 354 All 354 (JSON)