# Paraphrases do not explain word analogies

arXiv: https://arxiv.org/abs/2102.11749 · HTML: https://mlsanity.com/paper/2102.11749/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 6 of 6)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Analogies Explained: Towards Understanding Word Embeddings](https://mlsanity.com/paper/1901.09813/) (2019)
- [Towards Understanding Linear Word Analogies](https://mlsanity.com/paper/1810.04882/) (2018)
- [Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning](https://mlsanity.com/paper/1509.01692/) (2015)
- [Issues in evaluating semantic spaces using word analogies](https://mlsanity.com/paper/1606.07736/) (2016)
- [Analogies minus analogy test: measuring regularities in word embeddings](https://mlsanity.com/paper/2010.03446/) (2020)

## Built on by (most cited first, 2 of 2)

- [Paraphrase Types for Generation and Detection](https://mlsanity.com/paper/2310.14863/) (2023)
- [Contrastive Loss is All You Need to Recover Analogies as Parallel Lines](https://mlsanity.com/paper/2306.08221/) (2023)

## Abstract (arXiv)

Many types of distributional word embeddings (weakly) encode linguistic regularities as directions (the difference between "jump" and "jumped" will be in a similar direction to that of "walk" and "walked," and so on). Several attempts have been made to explain this fact. We respond to Allen and Hospedales' recent (ICML, 2019) theoretical explanation, which claims that word2vec and GloVe will encode linguistic regularities whenever a specific relation of paraphrase holds between the four words involved in the regularity. We demonstrate that the explanation does not go through: the paraphrase relations needed under this explanation do not hold empirically.