mlsanityData to 14 Oct 2026

Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performance

arXiv 2310.05597 ↗First posted 9 Oct 2023cs.CLcited by 9

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

While analogies are a common way to evaluate word embeddings in NLP, it is also of interest to investigate whether or not analogical reasoning is a task in itself that can be learned. In this paper, we test several ways to learn basic analogical reasoning, specifically focusing on analogies that are more typical of what is used to evaluate analogical reasoning in humans than those in commonly used NLP benchmarks. Our experiments find that models are able to learn analogical reasoning, even with a small amount of data. We additionally compare our models to a dataset with a human baseline, and find that after training, models approach human performance.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 17 of its 37 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

17 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Emergent Analogical Reasoning in Large Language Models19 Dec

Other references · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  5. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  6. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  7. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  8. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
  9. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
  10. 2017Analogical Inference for Multi-Relational Embeddingscited by 69
  11. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  12. 2022Problems with Cosine as a Measure of Embedding Similarity for High Frequency Wordscited by 49
4 more
  1. 2016Issues in evaluating semantic spaces using word analogiescited by 27
  2. 2021Distilling Relation Embeddings from Pre-trained Language Modelscited by 15
  3. 2022Richer Countries and Richer Representationscited by 7
  4. 2022Scientific and Creative Analogies in Pretrained Language Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 1; 2026: 2.

9 papers.

  1. 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Modelscited by 24
  2. 2023Do large language models solve verbal analogies like children do?cited by 13
  3. 2024Creativity in AI: Progresses and Challengescited by 7
  4. 2025Evaluating the Logical Reasoning Abilities of Large Reasoning Modelscited by 7
  5. 2024Quantifying analogy of concepts via ologs and wiring diagramscited by 2
  6. 2024Any four real numbers are on all fours with analogycited by 2
  7. 2024AILS-NTUA at SemEval-2024 Task 9: Cracking Brain Teasers: Transformer Models for Lateral Thinking Puzzlescited by 1
  8. 2026Can Large Language Models Generalize Procedures Across Representations?cited by 1
  9. 2026FrameNet Semantic Role Classification by Analogycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base
  2. 2024Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models
  3. 2024Evaluating the Robustness of Analogical Reasoning in Large Language Models
  4. 2024LLMs as Models for Analogical Reasoning
  5. 2024AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies
  6. 2025Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research
  7. 2025Modeling Understanding of Story-Based Analogies Using Large Language Models
  8. 2024Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?