# Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performance

arXiv: https://arxiv.org/abs/2310.05597 · HTML: https://mlsanity.com/paper/2310.05597/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Analogical Inference for Multi-Relational Embeddings](https://mlsanity.com/paper/1705.02426/) (2017)
- [BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?](https://mlsanity.com/paper/2105.04949/) (2021)
- [Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words](https://mlsanity.com/paper/2205.05092/) (2022)
- [Issues in evaluating semantic spaces using word analogies](https://mlsanity.com/paper/1606.07736/) (2016)
- [Distilling Relation Embeddings from Pre-trained Language Models](https://mlsanity.com/paper/2110.15705/) (2021)
- [Richer Countries and Richer Representations](https://mlsanity.com/paper/2205.05093/) (2022)
- [Scientific and Creative Analogies in Pretrained Language Models](https://mlsanity.com/paper/2211.15268/) (2022)

## Built on by (most cited first, 9 of 9)

- [KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models](https://mlsanity.com/paper/2407.17773/) (2024)
- [Do large language models solve verbal analogies like children do?](https://mlsanity.com/paper/2310.20384/) (2023)
- [Creativity in AI: Progresses and Challenges](https://mlsanity.com/paper/2410.17218/) (2024)
- [Evaluating the Logical Reasoning Abilities of Large Reasoning Models](https://mlsanity.com/paper/2505.11854/) (2025)
- [Quantifying analogy of concepts via ologs and wiring diagrams](https://mlsanity.com/paper/2402.01020/) (2024)
- [Any four real numbers are on all fours with analogy](https://mlsanity.com/paper/2407.18770/) (2024)
- [AILS-NTUA at SemEval-2024 Task 9: Cracking Brain Teasers: Transformer Models for Lateral Thinking Puzzles](https://mlsanity.com/paper/2404.01084/) (2024)
- [Can Large Language Models Generalize Procedures Across Representations?](https://mlsanity.com/paper/2602.03542/) (2026)
- [FrameNet Semantic Role Classification by Analogy](https://mlsanity.com/paper/2603.19825/) (2026)

## Abstract (arXiv)

While analogies are a common way to evaluate word embeddings in NLP, it is also of interest to investigate whether or not analogical reasoning is a task in itself that can be learned. In this paper, we test several ways to learn basic analogical reasoning, specifically focusing on analogies that are more typical of what is used to evaluate analogical reasoning in humans than those in commonly used NLP benchmarks. Our experiments find that models are able to learn analogical reasoning, even with a small amount of data. We additionally compare our models to a dataset with a human baseline, and find that after training, models approach human performance.