mlsanityData to 7 Oct 2026

Issues in evaluating semantic spaces using word analogies

arXiv 1606.07736 ↗First posted 24 Jun 2016cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

The offset method for solving word analogies has become a standard evaluation tool for vector-space semantic models: it is considered desirable for a space to represent semantic relations as consistent vector offsets. We show that the method's reliance on cosine similarity conflates offset consistency with largely irrelevant neighborhood structure, and propose simple baselines that should be used to improve the utility of the method in vector space evaluation.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 5; 2023: 6; 2024: 5; 2025: 2; 2026: 2.

26 papers.

Sort:

By area:

  1. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  2. 2023Linear Spaces of Meanings: Compositional Structures in Vision-Language Modelscited by 48
  3. 2022Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processescited by 22
  4. 2023In-Context Analogical Reasoning with Pre-Trained Language Modelscited by 22
  5. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 20
  6. 2021A comparative evaluation and analysis of three generations of Distributional Semantic Modelscited by 19
  7. 2021Idiosyncratic but not Arbitrary: Learning Idiolects in Online Registers Reveals Distinctive yet Consistent Individual Stylescited by 18
  8. 2024Understanding polysemanticity in neural networks through coding theorycited by 17
  9. 2021Distilling Relation Embeddings from Pre-trained Language Modelscited by 15
  10. 2021Probabilistic Analogical Mapping with Semantic Relation Networkscited by 14
  11. 2024Revisiting Word Embeddings in the LLM Eracited by 13
  12. 2023Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performancecited by 9
14 more
  1. 2024ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogiescited by 8
  2. 2022Evaluating the Construct Validity of Text Embeddings with Application to Survey Questionscited by 7
  3. 2023Construction Artifacts in Metaphor Identification Datasetscited by 5
  4. 2024Statistical Uncertainty in Word Embeddings: GloVe-Vcited by 4
  5. 2021Paraphrases do not explain word analogiescited by 2
  6. 2022Semantic properties of English nominal pluralization: Insights from word embeddingscited by 2
  7. 2023Contrastive Loss is All You Need to Recover Analogies as Parallel Linescited by 2
  8. 2024Any four real numbers are on all fours with analogycited by 2
  9. 2023RelBERT: Embedding Relations with Language Modelscited by 1
  10. 2022Beer2Vec : Extracting Flavors from Reviews for Thirst-Quenching Recommandationscited by 0
  11. 2022On the Information Content of Predictions in Word Analogy Testscited by 0
  12. 2025Revisiting Word Embeddings in the LLM Eracited by 0
  13. 2026Large Language Models provide support for the parallelogram theory of analogycited by 0
  14. 2026A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretabilitycited by 0