# Are all negatives created equal in contrastive instance discrimination?

arXiv: https://arxiv.org/abs/2010.06682 · HTML: https://mlsanity.com/paper/2010.06682/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 30 of 50)

- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [What to Hide from Your Students: Attention-Guided Masked Image Modeling](https://mlsanity.com/paper/2203.12719/) (2022)
- [An Empirical Study of Graph Contrastive Learning](https://mlsanity.com/paper/2109.01116/) (2021)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning](https://mlsanity.com/paper/2503.04812/) (2025)
- [TimesURL: Self-supervised Contrastive Learning for Universal Time Series Representation Learning](https://mlsanity.com/paper/2312.15709/) (2023)
- [Robust Contrastive Learning Using Negative Samples with Diminished Semantics](https://mlsanity.com/paper/2110.14189/) (2021)
- [A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives](https://mlsanity.com/paper/2102.12982/) (2021)
- [Solving Inefficiency of Self-supervised Representation Learning](https://mlsanity.com/paper/2104.08760/) (2021)
- [Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2109.01797/) (2021)
- [ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency](https://mlsanity.com/paper/2106.02342/) (2021)
- [CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning](https://mlsanity.com/paper/2212.10341/) (2022)
- [ProCo: Prototype-aware Contrastive Learning for Long-tailed Medical Image Classification](https://mlsanity.com/paper/2209.00183/) (2022)
- [CALDA: Improving Multi-Source Time Series Domain Adaptation with Contrastive Adversarial Learning](https://mlsanity.com/paper/2109.14778/) (2021)
- [Object-Aware Cropping for Self-Supervised Learning](https://mlsanity.com/paper/2112.00319/) (2021)
- [Understanding Contrastive Learning via Distributionally Robust Optimization](https://mlsanity.com/paper/2310.11048/) (2023)
- [Similarity Contrastive Estimation for Self-Supervised Soft Contrastive Learning](https://mlsanity.com/paper/2111.14585/) (2021)
- [SLIC: Self-Supervised Learning with Iterative Clustering for Human Action Videos](https://mlsanity.com/paper/2206.12534/) (2022)
- [COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation](https://mlsanity.com/paper/2210.01784/) (2022)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Hard Negative Sampling Strategies for Contrastive Representation Learning](https://mlsanity.com/paper/2206.01197/) (2022)
- [Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach](https://mlsanity.com/paper/2207.06267/) (2022)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [Understanding and Achieving Efficient Robustness with Adversarial Supervised Contrastive Learning](https://mlsanity.com/paper/2101.10027/) (2021)
- [Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations](https://mlsanity.com/paper/2103.12719/) (2021)
- [Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing](https://mlsanity.com/paper/2209.03268/) (2022)
- [Self-supervised debiasing using low rank regularization](https://mlsanity.com/paper/2210.05248/) (2022)
- [ViLTA: Enhancing Vision-Language Pre-training through Textual Augmentation](https://mlsanity.com/paper/2308.16689/) (2023)
- [Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning](https://mlsanity.com/paper/2111.03169/) (2021)
- [Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data](https://mlsanity.com/paper/2305.05208/) (2023)

## Abstract (arXiv)

Self-supervised learning has recently begun to rival supervised learning on computer vision tasks. Many of the recent approaches have been based on contrastive instance discrimination (CID), in which the network is trained to recognize two augmented versions of the same instance (a query and positive) while discriminating against a pool of other instances (negatives). The learned representation is then used on downstream tasks such as image classification. Using methodology from MoCo v2 (Chen et al., 2020), we divided negatives by their difficulty for a given query and studied which difficulty ranges were most important for learning useful representations. We found a minority of negatives -- the hardest 5% -- were both necessary and sufficient for the downstream task to reach nearly full accuracy. Conversely, the easiest 95% of negatives were unnecessary and insufficient. Moreover, the very hardest 0.1% of negatives were unnecessary and sometimes detrimental. Finally, we studied the properties of negatives that affect their hardness, and found that hard negatives were more semantically similar to the query, and that some negatives were more consistently easy or hard than we would expect by chance. Together, our results indicate that negatives vary in importance and that CID may benefit from more intelligent negative treatment.