# Do More Negative Samples Necessarily Hurt in Contrastive Learning?

arXiv: https://arxiv.org/abs/2205.01789 · HTML: https://mlsanity.com/paper/2205.01789/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 3 May 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Understanding self-supervised Learning Dynamics without Contrastive Pairs](https://mlsanity.com/paper/2102.06810/) (2021)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Intriguing Properties of Contrastive Losses](https://mlsanity.com/paper/2011.02803/) (2020)
- [Understanding Contrastive Learning Requires Incorporating Inductive Biases](https://mlsanity.com/paper/2202.14037/) (2022)
- [Investigating the Role of Negatives in Contrastive Representation Learning](https://mlsanity.com/paper/2106.09943/) (2021)
- [Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning](https://mlsanity.com/paper/2102.06866/) (2021)

## Built on by (most cited first, 30 of 39)

- [LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning](https://mlsanity.com/paper/2503.04812/) (2025)
- [SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes](https://mlsanity.com/paper/2211.11296/) (2022)
- [InfoNCE Loss Provably Learns Cluster-Preserving Representations](https://mlsanity.com/paper/2302.07920/) (2023)
- [Improving Self-Supervised Learning by Characterizing Idealized Representations](https://mlsanity.com/paper/2209.06235/) (2022)
- [Optimal Sample Complexity of Contrastive Learning](https://mlsanity.com/paper/2312.00379/) (2023)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Unsupervised Representation Learning for Time Series: A Review](https://mlsanity.com/paper/2308.01578/) (2023)
- [Contrastive Learning with Consistent Representations](https://mlsanity.com/paper/2302.01541/) (2023)
- [Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis](https://mlsanity.com/paper/2304.00395/) (2023)
- [Analysis of Using Sigmoid Loss for Contrastive Learning](https://mlsanity.com/paper/2402.12613/) (2024)
- [MSCDA: Multi-level Semantic-guided Contrast Improves Unsupervised Domain Adaptation for Breast MRI Segmentation in Small Datasets](https://mlsanity.com/paper/2301.02554/) (2023)
- [Headless Language Models: Learning without Predicting with Contrastive Weight Tying](https://mlsanity.com/paper/2309.08351/) (2023)
- [Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning](https://mlsanity.com/paper/2506.04411/) (2025)
- [FG-UAP: Feature-Gathering Universal Adversarial Perturbation](https://mlsanity.com/paper/2209.13113/) (2022)
- [Towards the Sparseness of Projection Head in Self-Supervised Learning](https://mlsanity.com/paper/2307.08913/) (2023)
- [Generalization Analysis for Supervised Contrastive Representation Learning under Non-IID Settings](https://mlsanity.com/paper/2505.04937/) (2025)
- [Table Integration in Data Lakes Unleashed: Pairwise Integrability Judgment, Integrable Set Discovery, and Multi-Tuple Conflict Resolution](https://mlsanity.com/paper/2412.00324/) (2024)
- [Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning](https://mlsanity.com/paper/2603.03530/) (2026)
- [Feature Normalization Prevents Collapse of Non-contrastive Learning Dynamics](https://mlsanity.com/paper/2309.16109/) (2023)
- [PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis](https://mlsanity.com/paper/2405.14650/) (2024)
- [On the Alignment Between Supervised and Self-Supervised Contrastive Learning](https://mlsanity.com/paper/2510.08852/) (2025)
- [DUEL: Adaptive Duplicate Elimination on Working Memory for Self-Supervised Learning](https://mlsanity.com/paper/2210.17052/) (2022)
- [DUEL: Duplicate Elimination on Active Memory for Self-Supervised Class-Imbalanced Learning](https://mlsanity.com/paper/2402.08963/) (2024)
- [Link Prediction with Physics-Inspired Graph Neural Networks](https://mlsanity.com/paper/2402.14802/) (2024)
- [Semantic Feature Learning for Universal Unsupervised Cross-Domain Retrieval](https://mlsanity.com/paper/2403.05690/) (2024)
- [Adv-SSL: Adversarial Self-Supervised Representation Learning with Theoretical Guarantees](https://mlsanity.com/paper/2408.08533/) (2024)
- [JojoSCL: Shrinkage Contrastive Learning for single-cell RNA sequence Clustering](https://mlsanity.com/paper/2506.00410/) (2025)
- [Enhancing Contrastive Learning-based Electrocardiogram Pretrained Model with Patient Memory Queue](https://mlsanity.com/paper/2506.06310/) (2025)
- [The Complexity of Finding Local Optima in Contrastive Learning](https://mlsanity.com/paper/2509.16898/) (2025)
- [Contrastive Learning for Inference in Dialogue](https://mlsanity.com/paper/2310.12467/) (2023)

## Abstract (arXiv)

Recent investigations in noise contrastive estimation suggest, both empirically as well as theoretically, that while having more "negative samples" in the contrastive loss improves downstream classification performance initially, beyond a threshold, it hurts downstream performance due to a "collision-coverage" trade-off. But is such a phenomenon inherent in contrastive learning? We show in a simple theoretical setting, where positive pairs are generated by sampling from the underlying latent class (introduced by Saunshi et al. (ICML 2019)), that the downstream performance of the representation optimizing the (population) contrastive loss in fact does not degrade with the number of negative samples. Along the way, we give a structural characterization of the optimal representation in our framework, for noise contrastive estimation. We also provide empirical support for our theoretical results on CIFAR-10 and CIFAR-100 datasets.