mlsanityData to 7 Oct 2026

Analogies minus analogy test: measuring regularities in word embeddings

arXiv 2010.03446 ↗First posted 7 Oct 2020cs.CL cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Vector space models of words have long been claimed to capture linguistic regularities as simple vector translations, but problems have been raised with this claim. We decompose and empirically analyze the classic arithmetic word analogy test, to motivate two new metrics that address the issues with the standard test, and which distinguish between class-wise offset concentration (similar directions between pairs of words drawn from different broad classes, such as France--London, China--Ottawa, ...) and pairing consistency (the existence of a regular transformation between correctly-matched pairs such as France:Paris::China:Beijing). We show that, while the standard analogy test is flawed, several popular word embeddings do nevertheless encode linguistic regularities.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 1; 2023: 3; 2024: 3; 2025: 5; 2026: 2.

17 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  2. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  3. 2021Embedding Arithmetic of Multimodal Queries for Image Retrievalcited by 21
  4. 2024Revisiting Word Embeddings in the LLM Eracited by 13
  5. 2023Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abductioncited by 11
  6. 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 11
  7. 2024Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models?cited by 3
  8. 2021Paraphrases do not explain word analogiescited by 2
  9. 2023Contrastive Loss is All You Need to Recover Analogies as Parallel Linescited by 2
  10. 2025Quantifying Compositionality of Classic and State-of-the-Art Embeddingscited by 2
  11. 2024Past Meets Present: Creating Historical Analogy with Large Language Modelscited by 1
  12. 2022On the Information Content of Predictions in Word Analogy Testscited by 0
5 more
  1. 2025Revisiting Word Embeddings in the LLM Eracited by 0
  2. 2025AIR: Zero-shot Generative Model Adaptation with Iterative Refinementcited by 0
  3. 2025Emphasis Sensitivity in Speech Representationscited by 0
  4. 2025Cross-modal Counterfactual Explanations: Uncovering Decision Factors and Dataset Biases in Subjective Classificationcited by 0
  5. 2026When Language Representations Interact: Separability and Cross-Lingual Effects in LLMscited by 0