mlsanityData to 7 Oct 2026

Hard Negative Sampling Strategies for Contrastive Representation Learning

arXiv 2206.01197 ↗First posted 2 Jun 2022cs.LG cs.AI cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

One of the challenges in contrastive learning is the selection of appropriate textithard negative examples, in the absence of label information. Random sampling or importance sampling methods based on feature similarity often lead to sub-optimal performance. In this work, we introduce UnReMix, a hard negative sampling strategy that takes into account anchor similarity, model uncertainty and representativeness. Experimental results on several benchmarks show that UnReMix improves negative sample selection, and subsequently downstream performance when compared to state-of-the-art contrastive learning methods.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 50 of its 61 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

50 of 61 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2018How Powerful are Graph Neural Networks?cited by 4,355
  10. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
38 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  3. 2019Contrastive Multiview Codingcited by 1,297
  4. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  6. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  7. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  8. 2020Contrastive Learning with Hard Negative Samplescited by 619
  9. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  10. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  11. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  12. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  13. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  14. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  15. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  16. 2020Debiased Contrastive Learningcited by 354
  17. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  18. 2015Skip-Thought Vectorscited by 321
  19. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  20. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  21. 2018An efficient framework for learning sentence representationscited by 197
  22. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 197
  23. 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
  24. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  25. 2020Boosting Contrastive Self-Supervised Learning with False Negative Cancellationcited by 119
  26. 2020AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversariescited by 90
  27. 2020Sequence-to-Sequence Contrastive Learning for Text Recognitioncited by 70
  28. 2020Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learningcited by 62
  29. 2020MixCo: Mix-up Contrastive Learning for Visual Representationcited by 62
  30. 2020Are all negatives created equal in contrastive instance discrimination?cited by 50
  31. 2021Improving Contrastive Learning by Visualizing Feature Transformationcited by 44
  32. 2021Robust Contrastive Learning Using Negative Samples with Diminished Semanticscited by 43
  33. 2021Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Predictioncited by 36
  34. 2021Contrastive Domain Adaptationcited by 36
  35. 2021Max-Margin Contrastive Learningcited by 27
  36. 2019Open Set Recognition Through Deep Neural Network Uncertainty: Does Out-of-Distribution Detection Require Generative Classifiers?cited by 13
  37. 2020EqCo: Equivalent Rules for Self-supervised Contrastive Learningcited by 12
  38. 2021Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Frameworkcited by 11

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 4; 2025: 2.

13 papers.

Sort:

By area:

  1. 2023Graph Contrastive Learning for Skeleton-based Action Recognitioncited by 17
  2. 2022Supervised Contrastive Learning with Hard Negative Samplescited by 13
  3. 2024Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Losscited by 10
  4. 2024Improved Diversity-Promoting Collaborative Metric Learning for Recommendationcited by 9
  5. 2022Self-supervised debiasing using low rank regularizationcited by 8
  6. 2025QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrievalcited by 8
  7. 2023BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphscited by 7
  8. 2024Contrastive Learning with Synthetic Positivescited by 6
  9. 2025Solving Instance Detection from an Open-World Perspectivecited by 6
  10. 2023Multimodal Contrastive Learning with Hard Negative Sampling for Human Activity Recognitioncited by 4
  11. 2023Text Representation Distillation via Information Bottleneck Principlecited by 3
  12. 2023Mitigating Pooling Bias in E-commerce Search via False Negative Estimationcited by 0
1 more
  1. 2024ManiNeg: Manifestation-guided Multimodal Pretraining for Mammography Classificationcited by 0