# Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning

arXiv: https://arxiv.org/abs/1509.01692 · HTML: https://mlsanity.com/paper/1509.01692/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2015.

## Built on by (most cited first, 29 of 29)

- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?](https://mlsanity.com/paper/2105.04949/) (2021)
- [Robust Retrieval Augmented Generation for Zero-shot Slot Filling](https://mlsanity.com/paper/2108.13934/) (2021)
- [Does ChatGPT Have a Mind?](https://mlsanity.com/paper/2407.11015/) (2024)
- [Concept Alignment](https://mlsanity.com/paper/2401.08672/) (2024)
- [Compression ensembles quantify aesthetic complexity and the evolution of visual art](https://mlsanity.com/paper/2205.10271/) (2022)
- [What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured](https://mlsanity.com/paper/2204.05673/) (2022)
- [SEGA: Instructing Text-to-Image Models using Semantic Guidance](https://mlsanity.com/paper/2301.12247/) (2023)
- [Hierarchies over Vector Space: Orienting Word and Graph Embeddings](https://mlsanity.com/paper/2211.01430/) (2022)
- [A Multidimensional Framework for Evaluating Lexical Semantic Change with Social Science Applications](https://mlsanity.com/paper/2406.06052/) (2024)
- [Leveraging Chemistry Foundation Models to Facilitate Structure Focused Retrieval Augmented Generation in Multi-Agent Workflows for Catalyst and Materials Design](https://mlsanity.com/paper/2408.11793/) (2024)
- [Paraphrases do not explain word analogies](https://mlsanity.com/paper/2102.11749/) (2021)
- [CO-NNECT: A Framework for Revealing Commonsense Knowledge Paths as Explicitations of Implicit Knowledge in Texts](https://mlsanity.com/paper/2105.03157/) (2021)
- [RelBERT: Embedding Relations with Language Models](https://mlsanity.com/paper/2310.00299/) (2023)
- [Semantic Guidance Tuning for Text-To-Image Diffusion Models](https://mlsanity.com/paper/2312.15964/) (2023)
- [KnowledgePrompts: Exploring the Abilities of Large Language Models to Solve Proportional Analogies via Knowledge-Enhanced Prompting](https://mlsanity.com/paper/2412.00869/) (2024)
- [Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory](https://mlsanity.com/paper/2505.14080/) (2025)
- [Linear Relational Decoding of Morphology in Language Models](https://mlsanity.com/paper/2507.14640/) (2025)
- [Disentangling Latent Embeddings with Sparse Linear Concept Subspaces (SLiCS)](https://mlsanity.com/paper/2508.20322/) (2025)
- [Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmetic](https://mlsanity.com/paper/2510.17001/) (2025)
- [An Evaluation Dataset for Legal Word Embedding: A Case Study On Chinese Codex](https://mlsanity.com/paper/2203.15173/) (2022)
- [Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences](https://mlsanity.com/paper/2205.07603/) (2022)
- [Beyond Model Interpretability: On the Faithfulness and Adversarial Robustness of Contrastive Textual Explanations](https://mlsanity.com/paper/2210.08902/) (2022)
- [Exploring Internal Numeracy in Language Models: A Case Study on ALBERT](https://mlsanity.com/paper/2404.16574/) (2024)
- [Optimal synthesis embeddings](https://mlsanity.com/paper/2406.10259/) (2024)
- [AIR: Zero-shot Generative Model Adaptation with Iterative Refinement](https://mlsanity.com/paper/2506.10895/) (2025)
- [Open-SAT: LLM-Guided Query Embedding Refinement for Open-Vocabulary Object Retrieval in Satellite Imagery](https://mlsanity.com/paper/2605.05344/) (2026)
- [CAST: Closed-form Analytic Semantic Transfer for Zero-Shot Classifier Extension](https://mlsanity.com/paper/2608.13751/) (2026)
- [Recovering Temporal and Geographic Signals from Language Model Embeddings](https://mlsanity.com/paper/2609.05721/) (2026)

## Abstract (arXiv)

Recent work on word embeddings has shown that simple vector subtraction over pre-trained embeddings is surprisingly effective at capturing different lexical relations, despite lacking explicit supervision. Prior work has evaluated this intriguing result using a word analogy prediction formulation and hand-selected relations, but the generality of the finding over a broader range of lexical relation types and different learning settings has not been evaluated. In this paper, we carry out such an evaluation in two learning settings: (1) spectral clustering to induce word relations, and (2) supervised learning to classify vector differences into relation types. We find that word embeddings capture a surprising amount of information, and that, under suitable supervised training, vector subtraction generalises well to a broad range of relations, including over unseen lexical items.