mlsanityData to 7 Oct 2026

Does Negative Sampling Matter? A Review with Insights into its Theory and Applications

arXiv 2402.17238 ↗First posted 27 Feb 2024cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Negative sampling has swiftly risen to prominence as a focal point of research, with wide-ranging applications spanning machine learning, computer vision, natural language processing, data mining, and recommender systems. This growing interest raises several critical questions: Does negative sampling really matter? Is there a general framework that can incorporate all existing negative sampling methods? In what fields is it applied? Addressing these questions, we propose a general framework that leverages negative sampling. Delving into the history of negative sampling, we trace the development of negative sampling through five evolutionary paths. We dissect and categorize the strategies used to select negative sample candidates, detailing global, local, mini-batch, hop, and memory-based approaches. Our review categorizes current negative sampling methods into five types: static, hard, GAN-based, Auxiliary-based, and In-batch methods, providing a clear structure for understanding negative sampling. Beyond detailed categorization, we highlight the application of negative sampling in various areas, offering insights into its practical benefits. Finally, we briefly discuss open problems and future directions for negative sampling.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 122 of its 184 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

122 of 184 references.

References · 122

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2017Graph Attention Networkscited by 8,854
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2017Inductive Representation Learning on Large Graphscited by 6,284
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
38 more
  1. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  5. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  10. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  11. 2019Contrastive Multiview Codingcited by 1,297
  12. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  13. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
  14. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  15. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
  16. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
  17. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  18. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  19. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  20. 2019Neural Graph Collaborative Filteringcited by 660
  21. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  22. 2020Contrastive Learning with Hard Negative Samplescited by 619
  23. 2015LINE: Large-scale Information Network Embeddingcited by 582
  24. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  25. 2020Deep Graph Contrastive Representation Learningcited by 535
  26. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  27. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  28. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  29. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  30. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  31. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  32. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  33. 2020Understanding the Behaviour of Contrastive Losscited by 397
  34. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  35. 2020Debiased Contrastive Learningcited by 354
  36. 2018Session-based Recommendation with Graph Neural Networkscited by 312
  37. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  38. 2020Self-supervised Graph Learning for Recommendationcited by 305

50 most cited of 122 All 122 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024Renovating Names in Open-Vocabulary Segmentation Benchmarkscited by 7
  2. 2025CLEF: Clinically-Guided Contrastive Learning for Electrocardiogram Foundation Modelscited by 7
  3. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4
  4. 2024Privately Learning from Graphs with Applications in Fine-tuning Large Language Modelscited by 1
  5. 2025Structure-preserving contrastive learning for spatial time seriescited by 1
  6. 2025Multiple Instance Learning Framework with Masked Hard Instance Mining for Gigapixel Histopathology Image Analysiscited by 1
  7. 2024Bypassing Skip-Gram Negative Sampling: Dimension Regularization as a More Efficient Alternative for Graph Embeddingscited by 0
  8. 2024Multi-Margin Cosine Loss: Proposal and Application in Recommender Systemscited by 0
  9. 2025Evaluating Negative Sampling Approaches for Neural Topic Modelscited by 0
  10. 2025Demystifying Distributed Training of Graph Neural Networks for Link Predictioncited by 0
  11. 2025A Graph-in-Graph Learning Framework for Drug-Target Interaction Predictioncited by 0
  12. 2026Sustainable Intelligence for the Wild: Democratizing Ecological Monitoring via Knowledge-Adaptive Edge Expert Agentscited by 0
2 more
  1. 2026MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel Detectioncited by 0
  2. 2026Multi-Functional Embedding Models for Funder Name Disambiguation in Scientific Publication Recordscited by 0