# Inductive reasoning in humans and large language models

arXiv: https://arxiv.org/abs/2306.06548 · HTML: https://mlsanity.com/paper/2306.06548/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 20 of its 76 references that are in the map.
- First posted on arXiv 11 Jun 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 20 of 20)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [Data Contamination: From Memorization to Exploitation](https://mlsanity.com/paper/2203.08242/) (2022)
- [Implicit Representations of Meaning in Neural Language Models](https://mlsanity.com/paper/2106.00737/) (2021)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches](https://mlsanity.com/paper/1904.01172/) (2019)
- [Word meaning in minds and machines](https://mlsanity.com/paper/2008.01766/) (2020)
- [Do language models learn typicality judgments from text?](https://mlsanity.com/paper/2105.02987/) (2021)
- [MEWL: Few-shot multimodal word learning with referential uncertainty](https://mlsanity.com/paper/2306.00503/) (2023)
- [Evaluating statistical language models as pragmatic reasoners](https://mlsanity.com/paper/2305.01020/) (2023)

## Built on by (most cited first, 24 of 24)

- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement](https://mlsanity.com/paper/2310.08559/) (2023)
- [Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey](https://mlsanity.com/paper/2404.01869/) (2024)
- [Automated Hypothesis Validation with Agentic Sequential Falsifications](https://mlsanity.com/paper/2502.09858/) (2025)
- [EXP-Bench: Can AI Conduct AI Research Experiments?](https://mlsanity.com/paper/2505.24785/) (2025)
- [Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.00836/) (2023)
- [Probing the limitations of multimodal language models for chemistry and materials research](https://mlsanity.com/paper/2411.16955/) (2024)
- [Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics](https://mlsanity.com/paper/2405.04669/) (2024)
- [(Ir)rationality and Cognitive Biases in Large Language Models](https://mlsanity.com/paper/2402.09193/) (2024)
- [A Philosophical Introduction to Language Models - Part II: The Way Forward](https://mlsanity.com/paper/2405.03207/) (2024)
- [Enhancing Genetic Improvement Mutations Using Large Language Models](https://mlsanity.com/paper/2310.19813/) (2023)
- [An Evaluation of Estimative Uncertainty in Large Language Models](https://mlsanity.com/paper/2405.15185/) (2024)
- [A Survey on Human-Centric LLMs](https://mlsanity.com/paper/2411.14491/) (2024)
- [From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models](https://mlsanity.com/paper/2407.09502/) (2024)
- [DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learning](https://mlsanity.com/paper/2505.03209/) (2025)
- [Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It](https://mlsanity.com/paper/2507.13328/) (2025)
- [Philosophy of Cognitive Science in the Age of Deep Learning](https://mlsanity.com/paper/2405.04048/) (2024)
- [GPT-ology, Computational Models, Silicon Sampling: How should we think about LLMs in Cognitive Science?](https://mlsanity.com/paper/2406.09464/) (2024)
- [Heuristic Reasoning in AI: Instrumental Use and Mimetic Absorption](https://mlsanity.com/paper/2403.09404/) (2024)
- [Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience](https://mlsanity.com/paper/2505.05515/) (2025)
- [On Language Models' Sensitivity to Suspicious Coincidences](https://mlsanity.com/paper/2504.09387/) (2025)
- [Predicate Renaming via Large Language Models](https://mlsanity.com/paper/2510.25517/) (2025)
- [Iconicity in Large Language Models](https://mlsanity.com/paper/2501.05643/) (2025)
- [Latent Taxonomies: How Property Induction Develops in Language Model Pre-training](https://mlsanity.com/paper/2610.15836/) (2026)

## Abstract (arXiv)

The impressive recent performance of large language models has led many to wonder to what extent they can serve as models of general intelligence or are similar to human cognition. We address this issue by applying GPT-3.5 and GPT-4 to a classic problem in human inductive reasoning known as property induction. Over two experiments, we elicit human judgments on a range of property induction tasks spanning multiple domains. Although GPT-3.5 struggles to capture many aspects of human behaviour, GPT-4 is much more successful: for the most part, its performance qualitatively matches that of humans, and the only notable exception is its failure to capture the phenomenon of premise non-monotonicity. Our work demonstrates that property induction allows for interesting comparisons between human and machine intelligence and provides two large datasets that can serve as benchmarks for future work in this vein.