mlsanityData to 7 Oct 2026

word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

arXiv 1402.3722 ↗First posted 15 Feb 2014cs.CL cs.LG stat.MLcited by 109

Not on the map. Why?

Abstract (arXiv)

The word2vec software of Tomas Mikolov and colleagues (https://code.google.com/p/word2vec/ ) has gained a lot of traction lately, and provides state-of-the-art word embeddings. The learning models behind the software are described in two research papers. We found the description of the models in these papers to be somewhat cryptic and hard to follow. While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to figure out the rationale behind the equations. This note is an attempt to explain equation (4) (negative sampling) in "Distributed Representations of Words and Phrases and their Compositionality" by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean.

Key facts. Cited by 109 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Feb 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 23; 2023: 17; 2024: 16; 2025: 14; 2026: 3.

109 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  2. 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 163
  3. 2021Graph Learning: A Surveycited by 140
  4. 2021ML-Decoder: Scalable and Versatile Classification Headcited by 78
  5. 2022Zero-Shot Temporal Action Detection via Vision-Language Promptingcited by 65
  6. 2023Concept Algebra for (Score-Based) Text-Controlled Generative Modelscited by 59
  7. 2021Molecule Generation by Principal Subgraph Mining and Assemblingcited by 53
  8. 2021Dynamic Network Embedding Surveycited by 44
  9. 2022TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twittercited by 44
  10. 2021Chemical-Reaction-Aware Molecule Representation Learningcited by 36
  11. 2022From t-SNE to UMAP with contrastive learningcited by 28
  12. 2024Universal Time-Series Representation Learning: A Surveycited by 28
38 more
  1. 2022Contrastive Learning with Hard Negative Entities for Entity Set Expansioncited by 26
  2. 2026Subliminal Effects in Your Data: A General Mechanism via Log-Linearitycited by 15
  3. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14
  4. 2023Backdooring Textual Inversion for Concept Censorshipcited by 14
  5. 2024Does Negative Sampling Matter? A Review with Insights into its Theory and Applicationscited by 14
  6. 2023ZEETAD: Adapting Pretrained Vision-Language Model for Zero-Shot End-to-End Temporal Action Detectioncited by 12
  7. 2024Operator SVD with Neural Networks via Nested Low-Rank Approximationcited by 12
  8. 2021Graph Neural Network to Dilute Outliers for Refactoring Monolith Applicationcited by 9
  9. 2021BERT-Based Sentiment Analysis: A Software Engineering Perspectivecited by 9
  10. 2021Contrastive Neural Processes for Self-Supervised Learningcited by 9
  11. 2022Neural Network based Formation of Cognitive Maps of Semantic Spaces and the Emergence of Abstract Conceptscited by 8
  12. 2025Sequences of Logits Reveal the Low Rank Structure of Language Modelscited by 8
  13. 2021Semantically Constrained Memory Allocation (SCMA) for Embedding in Efficient Recommendation Systemscited by 7
  14. 2021Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embeddingcited by 7
  15. 2021Data Augmentation for Mental Health Classification on Social Mediacited by 7
  16. 2022A Siren Song of Open Source Reproducibilitycited by 7
  17. 2023A Scalable Framework for Automatic Playlist Continuation on Music Streaming Servicescited by 7
  18. 2024Renovating Names in Open-Vocabulary Segmentation Benchmarkscited by 7
  19. 2025Finsler Multi-Dimensional Scaling: Manifold Learning for Asymmetric Dimensionality Reduction and Embeddingcited by 7
  20. 2021DeepWalking Backwards: From Embeddings Back to Graphscited by 6
  21. 2021Drug Recommendation System based on Sentiment Analysis of Drug Reviews using Machine Learningcited by 6
  22. 2022The Document Vectors Using Cosine Similarity Revisitedcited by 5
  23. 2024Probing the contents of semantic representations from text, behavior, and brain data using the psychNorms metabasecited by 5
  24. 2025Adversarial Attacks on AI-Generated Text Detection Models: A Token Probability-Based Approach Using Embeddingscited by 5
  25. 2021Theoretical Understandings of Product Embedding for E-commerce Machine Learningcited by 4
  26. 2021quantum Case-Based Reasoning (qCBR)cited by 4
  27. 2021Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learningcited by 4
  28. 2022kNN-Embed: Locally Smoothed Embedding Mixtures For Multi-interest Candidate Retrievalcited by 4
  29. 2023From Characters to Words: Hierarchical Pre-trained Language Model for Open-vocabulary Language Understandingcited by 4
  30. 2024LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logscited by 4
  31. 2024Exploring State Space and Reasoning by Elimination in Tsetlin Machinescited by 4
  32. 2021Network Representation Learning: From Traditional Feature Learning to Deep Learningcited by 3
  33. 2021Should Semantic Vector Composition be Explicit? Can it be Linear?cited by 3
  34. 2021ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrievalcited by 3
  35. 2022Word Embedding for Social Sciences: An Interdisciplinary Surveycited by 3
  36. 2023Detection of Fake Generated Scientific Abstractscited by 3
  37. 2025How Low Can You Go? Searching for the Intrinsic Dimensionality of Complex Networks using Metric Node Embeddingscited by 3
  38. 2021A multi-modal approach towards mining social media data during natural disasters -- a case study of Hurricane Irmacited by 2

50 most cited of 109 All 109 (JSON)