# Do Large Language Models know what humans know?

arXiv: https://arxiv.org/abs/2209.01515 · HTML: https://mlsanity.com/paper/2209.01515/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 80 references that are in the map.
- First posted on arXiv 4 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [Probing Neural Network Comprehension of Natural Language Arguments](https://mlsanity.com/paper/1907.07355/) (2019)
- [Implicit Representations of Meaning in Neural Language Models](https://mlsanity.com/paper/2106.00737/) (2021)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [What Artificial Neural Networks Can Tell Us About Human Language Acquisition](https://mlsanity.com/paper/2208.07998/) (2022)
- [Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations](https://mlsanity.com/paper/2109.14989/) (2021)
- [RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency](https://mlsanity.com/paper/1809.01329/) (2018)
- [So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgements](https://mlsanity.com/paper/2109.01226/) (2021)
- [The Singleton Fallacy: Why Current Critiques of Language Models Miss the Point](https://mlsanity.com/paper/2102.04310/) (2021)
- [RAW-C: Relatedness of Ambiguous Words--in Context (A New Lexical Resource for English)](https://mlsanity.com/paper/2105.13266/) (2021)

## Built on by (most cited first, 30 of 60)

- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Understanding Social Reasoning in Language Models with Language Models](https://mlsanity.com/paper/2306.15448/) (2023)
- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [How FaR Are Large Language Models From Agents with Theory-of-Mind?](https://mlsanity.com/paper/2310.03051/) (2023)
- [Boosting Theory-of-Mind Performance in Large Language Models via Prompting](https://mlsanity.com/paper/2304.11490/) (2023)
- [SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs](https://mlsanity.com/paper/2410.13648/) (2024)
- [Event knowledge in large language models: the gap between the impossible and the unlikely](https://mlsanity.com/paper/2212.01488/) (2022)
- [Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models](https://mlsanity.com/paper/2310.19619/) (2023)
- [AI for social science and social science of AI: A Survey](https://mlsanity.com/paper/2401.11839/) (2024)
- [The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agents](https://mlsanity.com/paper/2307.07871/) (2023)
- [Re-evaluating Theory of Mind evaluation in large language models](https://mlsanity.com/paper/2502.21098/) (2025)
- [The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?](https://mlsanity.com/paper/2305.07970/) (2023)
- [Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models](https://mlsanity.com/paper/2407.06004/) (2024)
- [Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding](https://mlsanity.com/paper/2310.19671/) (2023)
- [Unveiling Theory of Mind in Large Language Models: A Parallel to Single Neurons in the Human Brain](https://mlsanity.com/paper/2309.01660/) (2023)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models](https://mlsanity.com/paper/2412.17128/) (2024)
- [Stick to your Role! Stability of Personal Values Expressed in Large Language Models](https://mlsanity.com/paper/2402.14846/) (2024)
- [Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs](https://mlsanity.com/paper/2401.04854/) (2024)
- [Can Large Language Models Help Experimental Design for Causal Discovery?](https://mlsanity.com/paper/2503.01139/) (2025)
- [AI with Emotions: Exploring Emotional Expressions in Large Language Models](https://mlsanity.com/paper/2504.14706/) (2025)
- [Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models](https://mlsanity.com/paper/2510.04764/) (2025)
- [Epistemological Fault Lines Between Human and Artificial Intelligence](https://mlsanity.com/paper/2512.19466/) (2025)
- [Recent Advancement of Emotion Cognition in Large Language Models](https://mlsanity.com/paper/2409.13354/) (2024)
- [Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task](https://mlsanity.com/paper/2507.16196/) (2025)
- [Using Artificial Populations to Study Psychological Phenomena in Neural Models](https://mlsanity.com/paper/2308.08032/) (2023)
- [Do Large Language Models Learn Human-Like Strategic Preferences?](https://mlsanity.com/paper/2404.08710/) (2024)
- [HLB: Benchmarking LLMs' Humanlikeness in Language Use](https://mlsanity.com/paper/2409.15890/) (2024)
- [Dissecting the Ullman Variations with a SCALPEL: Why do LLMs fail at Trivial Alterations to the False Belief Task?](https://mlsanity.com/paper/2406.14737/) (2024)

## Abstract (arXiv)

Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre-registered analyses, we present a linguistic version of the False Belief Task to both human participants and a Large Language Model, GPT-3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans, nor does it explain the full extent of their behavior -- despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible.