# Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations

arXiv: https://arxiv.org/abs/1606.00819 · HTML: https://mlsanity.com/paper/1606.00819/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2016.

## Built on by (most cited first, 5 of 5)

- [Probing the contents of semantic representations from text, behavior, and brain data using the psychNorms metabase](https://mlsanity.com/paper/2412.04936/) (2024)
- [Linguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskrit](https://mlsanity.com/paper/2308.08807/) (2023)
- [One Word Is Not Enough: Simple Prompts Improve Word Embeddings](https://mlsanity.com/paper/2512.06744/) (2025)
- [A Topological Approach to Compare Document Semantics Based on a New Variant of Syntactic N-grams](https://mlsanity.com/paper/2103.05135/) (2021)
- [NoPPA: Non-Parametric Pairwise Attention Random Walk Model for Sentence Representation](https://mlsanity.com/paper/2302.12903/) (2023)

## Abstract (arXiv)

In this paper, we propose LexVec, a new method for generating distributed word representations that uses low-rank, weighted factorization of the Positive Point-wise Mutual Information matrix via stochastic gradient descent, employing a weighting scheme that assigns heavier penalties for errors on frequent co-occurrences while still accounting for negative co-occurrence. Evaluation on word similarity and analogy tasks shows that LexVec matches and often outperforms state-of-the-art methods on many of these tasks.