mlsanityData to 7 Oct 2026

Emergent Analogical Reasoning in Large Language Models

arXiv 2212.09196 ↗First posted 19 Dec 2022cs.AI cs.CLcited by 239

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The recent advent of large language models has reinvigorated debate over whether human cognitive capacities might emerge in such generic models given sufficient training data. Of particular interest is the ability of these models to reason about novel problems zero-shot, without any direct training. In human cognition, this capacity is closely tied to an ability to reason by analogy. Here, we performed a direct comparison between human reasoners and a large language model (the text-davinci-003 variant of GPT-3) on a range of analogical tasks, including a non-visual matrix reasoning task based on the rule structure of Raven's Standard Progressive Matrices. We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings; preliminary tests of GPT-4 indicated even better performance. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 25 of its 93 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

25 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language models show human-like content effects on reasoning tasks14 Jul

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  9. 2022Using cognitive psychology to understand GPT-3cited by 389
  10. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 303
  11. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  12. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
12 more
  1. 2018Measuring abstract reasoning in neural networkscited by 165
  2. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
  3. 2022A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matricescited by 64
  4. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  5. 2019Learning to Make Analogies by Contrasting Abstract Relational Structurecited by 53
  6. 2019SciPy 1.0--Fundamental Algorithms for Scientific Computing in Pythoncited by 49
  7. 2020Emergent Symbols through Binding in External Memorycited by 47
  8. 2020The Scattering Compositional Learner: Discovering Objects, Attributes, Relationships in Analogical Reasoningcited by 46
  9. 2020Learning Representations that Support Extrapolationcited by 36
  10. 2020Understanding Human Intelligence through Human Limitationscited by 33
  11. 2021Probabilistic Analogical Mapping with Semantic Relation Networkscited by 14
  12. 2022Zero-shot visual reasoning through probabilistic analogical mappingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 62; 2024: 77; 2025: 59; 2026: 41.

239 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2023Playing repeated games with Large Language Modelscited by 213
  5. 2023Large Language Models as General Pattern Machinescited by 196
  6. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
  7. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  8. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 137
  9. 2023Large Language Models for Robotics: A Surveycited by 132
  10. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  11. 2023Teaching Arithmetic to Small Transformerscited by 121
  12. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
38 more
  1. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  2. 2023Large Language Models as Analogical Reasonerscited by 107
  3. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  4. 2024ChemLLM: A Chemical Large Language Modelcited by 100
  5. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 97
  6. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 94
  7. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  8. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  9. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  10. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  11. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  12. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  13. 2023Turning large language models into cognitive modelscited by 66
  14. 2023Meta-in-context learning in large language modelscited by 59
  15. 2023Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Taskscited by 54
  16. 2024PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patternscited by 54
  17. 2024Auxiliary task demands mask the capabilities of smaller language modelscited by 54
  18. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  19. 2023Large language models predict human sensory judgments across six modalitiescited by 47
  20. 2023Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5cited by 41
  21. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
  22. 2023Visual cognition in multimodal large language modelscited by 38
  23. 2024Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Modelscited by 38
  24. 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Modelscited by 37
  25. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  26. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  27. 2024The broader spectrum of in-context learningcited by 33
  28. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  29. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  30. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 31
  31. 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Modelscited by 31
  32. 2024A phase transition between positional and semantic learning in a solvable model of dot-product attentioncited by 30
  33. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
  34. 2024AI for social science and social science of AI: A Surveycited by 29
  35. 2025Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modelingcited by 29
  36. 2023The Relational Bottleneck as an Inductive Bias for Efficient Abstractioncited by 26
  37. 2024Evaluating Human-AI Collaboration: A Review and Methodological Frameworkcited by 25
  38. 2023LLMind: Orchestrating AI and IoT with LLM for Complex Task Executioncited by 24

50 most cited of 239 All 239 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating the Robustness of Analogical Reasoning in Large Language Models
  2. 2024AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies
  3. 2023ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base
  4. 2024Inductive Linguistic Reasoning with Large Language Models
  5. 2022Understanding Narratives through Dimensions of Analogy
  6. 2022Scientific and Creative Analogies in Pretrained Language Models
  7. 2022E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning
  8. 2025Large Language and Reasoning Models are Shallow Disjunctive Reasoners