# Conditional Negative Sampling for Contrastive Learning of Visual Representations

arXiv: https://arxiv.org/abs/2010.02037 · HTML: https://mlsanity.com/paper/2010.02037/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 30 of 57)

- [Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations](https://mlsanity.com/paper/2203.01517/) (2022)
- [An Empirical Study of Graph Contrastive Learning](https://mlsanity.com/paper/2109.01116/) (2021)
- [Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.09025/) (2021)
- [CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP](https://mlsanity.com/paper/2110.11316/) (2021)
- [ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning](https://mlsanity.com/paper/2110.02027/) (2021)
- [Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning](https://mlsanity.com/paper/2107.01886/) (2021)
- [Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo](https://mlsanity.com/paper/2203.17248/) (2022)
- [Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning](https://mlsanity.com/paper/2102.06605/) (2021)
- [Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2109.01797/) (2021)
- [CLEVE: Contrastive Pre-training for Event Extraction](https://mlsanity.com/paper/2105.14485/) (2021)
- [Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning](https://mlsanity.com/paper/2106.02866/) (2021)
- [Temperature as Uncertainty in Contrastive Learning](https://mlsanity.com/paper/2110.04403/) (2021)
- [Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization](https://mlsanity.com/paper/2305.11965/) (2023)
- [Conditional Contrastive Learning with Kernel](https://mlsanity.com/paper/2202.05458/) (2022)
- [Similarity Contrastive Estimation for Self-Supervised Soft Contrastive Learning](https://mlsanity.com/paper/2111.14585/) (2021)
- [Contrastive Reinforcement Learning of Symbolic Reasoning Domains](https://mlsanity.com/paper/2106.09146/) (2021)
- [GraphSHA: Synthesizing Harder Samples for Class-Imbalanced Node Classification](https://mlsanity.com/paper/2306.09612/) (2023)
- [Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders](https://mlsanity.com/paper/2212.07050/) (2022)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals](https://mlsanity.com/paper/2107.14762/) (2021)
- [Contrastive Attraction and Contrastive Repulsion for Representation Learning](https://mlsanity.com/paper/2105.03746/) (2021)
- [Supervised Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2209.00078/) (2022)
- [The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learning](https://mlsanity.com/paper/2307.10907/) (2023)
- [GRIT-VLP: Grouped Mini-batch Sampling for Efficient Vision and Language Pre-training](https://mlsanity.com/paper/2208.04060/) (2022)
- [On the Pros and Cons of Momentum Encoder in Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2208.05744/) (2022)
- [Affinity Uncertainty-based Hard Negative Mining in Graph Contrastive Learning](https://mlsanity.com/paper/2301.13340/) (2023)
- [Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations](https://mlsanity.com/paper/2103.12719/) (2021)
- [A Low Rank Promoting Prior for Unsupervised Contrastive Learning](https://mlsanity.com/paper/2108.02696/) (2021)
- [BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphs](https://mlsanity.com/paper/2306.03355/) (2023)
- [Unsupervised Voice-Face Representation Learning by Cross-Modal Prototype Contrast](https://mlsanity.com/paper/2204.14057/) (2022)

## Abstract (arXiv)

Recent methods for learning unsupervised visual representations, dubbed contrastive learning, optimize the noise-contrastive estimation (NCE) bound on mutual information between two views of an image. NCE uses randomly sampled negative examples to normalize the objective. In this paper, we show that choosing difficult negatives, or those more similar to the current instance, can yield stronger representations. To do this, we introduce a family of mutual information estimators that sample negatives conditionally -- in a "ring" around each positive. We prove that these estimators lower-bound mutual information, with higher bias but lower variance than NCE. Experimentally, we find our approach, applied on top of existing models (IR, CMC, and MoCo) improves accuracy by 2-5% points in each case, measured by linear evaluation on four standard image datasets. Moreover, we find continued benefits when transferring features to a variety of new image distributions from the Meta-Dataset collection and to a variety of downstream tasks such as object detection, instance segmentation, and keypoint detection.