# In-Context Analogical Reasoning with Pre-Trained Language Models

arXiv: https://arxiv.org/abs/2305.17626 · HTML: https://mlsanity.com/paper/2305.17626/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 28 May 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)
- [Stratified Rule-Aware Network for Abstract Visual Reasoning](https://mlsanity.com/paper/2002.06838/) (2020)
- [Learning Perceptual Inference by Contrasting](https://mlsanity.com/paper/1912.00086/) (2019)
- [Learning to Make Analogies by Contrasting Abstract Relational Structure](https://mlsanity.com/paper/1902.00120/) (2019)
- [Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution](https://mlsanity.com/paper/2103.14230/) (2021)
- [Abstract Visual Reasoning with Tangram Shapes](https://mlsanity.com/paper/2211.16492/) (2022)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning](https://mlsanity.com/paper/2203.08480/) (2022)
- [Issues in evaluating semantic spaces using word analogies](https://mlsanity.com/paper/1606.07736/) (2016)
- [V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices](https://mlsanity.com/paper/1907.12271/) (2019)
- [Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processes](https://mlsanity.com/paper/2210.12197/) (2022)
- [VASR: Visual Analogies of Situation Recognition](https://mlsanity.com/paper/2212.04542/) (2022)
- [Few-shot Visual Reasoning with Meta-analogical Contrastive Learning](https://mlsanity.com/paper/2007.12020/) (2020)
- [Draw Me a Flower: Processing and Grounding Abstraction in Natural Language](https://mlsanity.com/paper/2106.14321/) (2021)
- [Evaluating Understanding on Conceptual Abstraction Benchmarks](https://mlsanity.com/paper/2206.14187/) (2022)

## Built on by (most cited first, 22 of 22)

- [Large Language Models as Analogical Reasoners](https://mlsanity.com/paper/2310.01714/) (2023)
- [The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning](https://mlsanity.com/paper/2504.05081/) (2025)
- [The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Models](https://mlsanity.com/paper/2401.12117/) (2024)
- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [LLMs as Models for Analogical Reasoning](https://mlsanity.com/paper/2406.13803/) (2024)
- [A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Education](https://mlsanity.com/paper/2410.21287/) (2024)
- [Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale](https://mlsanity.com/paper/2604.18572/) (2026)
- [Probabilistic Abduction for Visual Abstract Reasoning via Learning Rules in Vector-symbolic Architectures](https://mlsanity.com/paper/2401.16024/) (2024)
- [Towards Learning Abductive Reasoning using VSA Distributed Representations](https://mlsanity.com/paper/2406.19121/) (2024)
- [Do Large Language Models Solve ARC Visual Analogies Like People Do?](https://mlsanity.com/paper/2403.09734/) (2024)
- [Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning](https://mlsanity.com/paper/2412.05586/) (2024)
- [Can Large Reasoning Models do Analogical Reasoning under Perceptual Uncertainty?](https://mlsanity.com/paper/2503.11207/) (2025)
- [Experience Transfer for Multimodal LLM Agents in Minecraft Game](https://mlsanity.com/paper/2604.05533/) (2026)
- [I-RAVEN-X: Benchmarking Generalization and Robustness of Analogical and Mathematical Reasoning in Large Language and Reasoning Models](https://mlsanity.com/paper/2510.17496/) (2025)
- [Humanlike Cognitive Patterns as Emergent Phenomena in Large Language Models](https://mlsanity.com/paper/2412.15501/) (2024)
- [Funny-Valen-Tine: Planning Solution Distribution Enhances Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2407.02688/) (2024)
- [ANVIL: Analogies and Videos for Lecturers](https://mlsanity.com/paper/2605.16295/) (2026)
- [Analogical Proportions and Creativity: A Preliminary Study](https://mlsanity.com/paper/2310.13500/) (2023)
- [Johnny: Structuring Representation Space to Enhance Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2506.01970/) (2025)
- [A Study of Rule Omission in Raven's Progressive Matrices](https://mlsanity.com/paper/2510.03127/) (2025)
- [VisAnalog: A Diagnostic Suite for Visual Concept Transfer on Natural Images](https://mlsanity.com/paper/2605.23141/) (2026)
- [On the use of foundation models in cognitive science](https://mlsanity.com/paper/2608.07812/) (2026)

## Abstract (arXiv)

Analogical reasoning is a fundamental capacity of human cognition that allows us to reason abstractly about novel situations by relating them to past experiences. While it is thought to be essential for robust reasoning in AI systems, conventional approaches require significant training and/or hard-coding of domain knowledge to be applied to benchmark tasks. Inspired by cognitive science research that has found connections between human language and analogy-making, we explore the use of intuitive language-based abstractions to support analogy in AI systems. Specifically, we apply large pre-trained language models (PLMs) to visual Raven's Progressive Matrices (RPM), a common relational reasoning test. By simply encoding the perceptual features of the problem into language form, we find that PLMs exhibit a striking capacity for zero-shot relational reasoning, exceeding human performance and nearing supervised vision-based methods. We explore different encodings that vary the level of abstraction over task features, finding that higher-level abstractions further strengthen PLMs' analogical reasoning. Our detailed analysis reveals insights on the role of model complexity, in-context learning, and prior knowledge in solving RPM tasks.