# Analogies minus analogy test: measuring regularities in word embeddings

arXiv: https://arxiv.org/abs/2010.03446 · HTML: https://mlsanity.com/paper/2010.03446/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 17 of 17)

- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?](https://mlsanity.com/paper/2105.04949/) (2021)
- [Embedding Arithmetic of Multimodal Queries for Image Retrieval](https://mlsanity.com/paper/2112.03162/) (2021)
- [Revisiting Word Embeddings in the LLM Era](https://mlsanity.com/paper/2402.11094/) (2024)
- [[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic](https://mlsanity.com/paper/2602.18899/) (2026)
- [Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction](https://mlsanity.com/paper/2305.12660/) (2023)
- [Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models?](https://mlsanity.com/paper/2406.11375/) (2024)
- [Paraphrases do not explain word analogies](https://mlsanity.com/paper/2102.11749/) (2021)
- [Contrastive Loss is All You Need to Recover Analogies as Parallel Lines](https://mlsanity.com/paper/2306.08221/) (2023)
- [Quantifying Compositionality of Classic and State-of-the-Art Embeddings](https://mlsanity.com/paper/2509.19332/) (2025)
- [Past Meets Present: Creating Historical Analogy with Large Language Models](https://mlsanity.com/paper/2409.14820/) (2024)
- [On the Information Content of Predictions in Word Analogy Tests](https://mlsanity.com/paper/2210.09972/) (2022)
- [Revisiting Word Embeddings in the LLM Era](https://mlsanity.com/paper/2502.19607/) (2025)
- [AIR: Zero-shot Generative Model Adaptation with Iterative Refinement](https://mlsanity.com/paper/2506.10895/) (2025)
- [Emphasis Sensitivity in Speech Representations](https://mlsanity.com/paper/2508.11566/) (2025)
- [Cross-modal Counterfactual Explanations: Uncovering Decision Factors and Dataset Biases in Subjective Classification](https://mlsanity.com/paper/2512.18864/) (2025)
- [When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs](https://mlsanity.com/paper/2606.14347/) (2026)

## Abstract (arXiv)

Vector space models of words have long been claimed to capture linguistic regularities as simple vector translations, but problems have been raised with this claim. We decompose and empirically analyze the classic arithmetic word analogy test, to motivate two new metrics that address the issues with the standard test, and which distinguish between class-wise offset concentration (similar directions between pairs of words drawn from different broad classes, such as France--London, China--Ottawa, ...) and pairing consistency (the existence of a regular transformation between correctly-matched pairs such as France:Paris::China:Beijing). We show that, while the standard analogy test is flawed, several popular word embeddings do nevertheless encode linguistic regularities.