# word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

arXiv: https://arxiv.org/abs/1402.3722 · HTML: https://mlsanity.com/paper/1402.3722/

## Key facts

- Cited by 112 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2014.

## Built on by (most cited first, 30 of 112)

- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design](https://mlsanity.com/paper/2311.13743/) (2023)
- [Graph Learning: A Survey](https://mlsanity.com/paper/2105.00696/) (2021)
- [ML-Decoder: Scalable and Versatile Classification Head](https://mlsanity.com/paper/2111.12933/) (2021)
- [Zero-Shot Temporal Action Detection via Vision-Language Prompting](https://mlsanity.com/paper/2207.08184/) (2022)
- [Concept Algebra for (Score-Based) Text-Controlled Generative Models](https://mlsanity.com/paper/2302.03693/) (2023)
- [Molecule Generation by Principal Subgraph Mining and Assembling](https://mlsanity.com/paper/2106.15098/) (2021)
- [TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter](https://mlsanity.com/paper/2209.07562/) (2022)
- [Dynamic Network Embedding Survey](https://mlsanity.com/paper/2103.15447/) (2021)
- [Chemical-Reaction-Aware Molecule Representation Learning](https://mlsanity.com/paper/2109.09888/) (2021)
- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [From t-SNE to UMAP with contrastive learning](https://mlsanity.com/paper/2206.01816/) (2022)
- [Contrastive Learning with Hard Negative Entities for Entity Set Expansion](https://mlsanity.com/paper/2204.07789/) (2022)
- [Subliminal Effects in Your Data: A General Mechanism via Log-Linearity](https://mlsanity.com/paper/2602.04863/) (2026)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [Backdooring Textual Inversion for Concept Censorship](https://mlsanity.com/paper/2308.10718/) (2023)
- [Operator SVD with Neural Networks via Nested Low-Rank Approximation](https://mlsanity.com/paper/2402.03655/) (2024)
- [ZEETAD: Adapting Pretrained Vision-Language Model for Zero-Shot End-to-End Temporal Action Detection](https://mlsanity.com/paper/2311.00729/) (2023)
- [Graph Neural Network to Dilute Outliers for Refactoring Monolith Application](https://mlsanity.com/paper/2102.03827/) (2021)
- [BERT-Based Sentiment Analysis: A Software Engineering Perspective](https://mlsanity.com/paper/2106.02581/) (2021)
- [Contrastive Neural Processes for Self-Supervised Learning](https://mlsanity.com/paper/2110.13623/) (2021)
- [Sequences of Logits Reveal the Low Rank Structure of Language Models](https://mlsanity.com/paper/2510.24966/) (2025)
- [A Siren Song of Open Source Reproducibility](https://mlsanity.com/paper/2204.04372/) (2022)
- [Neural Network based Formation of Cognitive Maps of Semantic Spaces and the Emergence of Abstract Concepts](https://mlsanity.com/paper/2210.16062/) (2022)
- [Finsler Multi-Dimensional Scaling: Manifold Learning for Asymmetric Dimensionality Reduction and Embedding](https://mlsanity.com/paper/2503.18010/) (2025)
- [Semantically Constrained Memory Allocation (SCMA) for Embedding in Efficient Recommendation Systems](https://mlsanity.com/paper/2103.06124/) (2021)
- [Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embedding](https://mlsanity.com/paper/2106.07250/) (2021)
- [Data Augmentation for Mental Health Classification on Social Media](https://mlsanity.com/paper/2112.10064/) (2021)
- [A Scalable Framework for Automatic Playlist Continuation on Music Streaming Services](https://mlsanity.com/paper/2304.09061/) (2023)

## Abstract (arXiv)

The word2vec software of Tomas Mikolov and colleagues (https://code.google.com/p/word2vec/ ) has gained a lot of traction lately, and provides state-of-the-art word embeddings. The learning models behind the software are described in two research papers. We found the description of the models in these papers to be somewhat cryptic and hard to follow. While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to figure out the rationale behind the equations. This note is an attempt to explain equation (4) (negative sampling) in "Distributed Representations of Words and Phrases and their Compositionality" by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean.