mlsanityData to 13 Oct 2026

Evaluating Large Language Models in Theory of Mind Tasks

arXiv 2302.02083 ↗First posted 4 Feb 2023cs.CL cs.CY cs.HCcited by 218

On the map: World models → World models for language agents

Abstract (arXiv)

Eleven Large Language Models (LLMs) were assessed using a custom-made battery of false-belief tasks, considered a gold standard in testing Theory of Mind (ToM) in humans. The battery included 640 prompts spread across 40 diverse tasks, each one including a false-belief scenario, three closely matched true-belief control scenarios, and the reversed versions of all four. To solve a single task, a model needed to correctly answer 16 prompts across all eight scenarios. Smaller and older models solved no tasks; GPT-3-davinci-003 (from November 2022) and ChatGPT-3.5-turbo (from March 2023) solved 20% of the tasks; ChatGPT-4 (from June 2023) solved 75% of the tasks, matching the performance of six-year-old children observed in past studies. We explore the potential interpretation of these findings, including the intriguing possibility that ToM, previously considered exclusive to humans, may have spontaneously emerged as a byproduct of LLMs' improving language skills.

Key facts. Cited by 218 papers in the map (data through 13 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 11 of its 88 references that are in the map. First posted on arXiv 4 Feb 2023.

Builds on

11 of 88 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs24 Oct
  2. 2020Language Models are Few-Shot Learners28 May

Other references · 9

  1. 2017Attention Is All You Needcited by 60,106
  2. 2022Training language models to follow instructions with human feedbackcited by 15,856
  3. 2022Emergent Abilities of Large Language Modelscited by 2,499
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,978
  5. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  6. 2022Thinking Fast and Slow in Large Language Modelscited by 136
  7. 2018Evaluating Theory of Mind in Question Answeringcited by 64
  8. 2022Do Large Language Models know what humans know?cited by 60
  9. 2022Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal Viewcited by 31

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 15; 2024: 43; 2025: 80; 2026: 80.

218 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,433
  2. 2023Playing repeated games with Large Language Modelscited by 216
  3. 2023ChatGPT: Jack of all trades, master of nonecited by 204
  4. 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 183
  5. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  6. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  7. 2025Agentic Large Language Models, a surveycited by 107
  8. 2025LLM Social Simulations Are a Promising Research Methodcited by 98
  9. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  10. 2024How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environmentscited by 66
  11. 2024Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theorycited by 60
  12. 2024LLMs achieve adult human performance on higher-order theory of mind taskscited by 59
38 more
  1. 2023On the Creativity of Large Language Modelscited by 57
  2. 2024Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Modelscited by 55
  3. 2024Large Language Models Assume People are More Rational than We Really arecited by 53
  4. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 51
  5. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agentscited by 48
  6. 2024Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Modelscited by 47
  7. 2025Language Models use Lookbacks to Track Beliefscited by 36
  8. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  9. 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 35
  10. 2025Large Language Models Miss the Multi-Agent Markcited by 26
  11. 2025On Benchmarking Human-Like Intelligence in Machinescited by 24
  12. 2025Theory of Mind in Large Language Models: Assessment and Enhancementcited by 24
  13. 2025Re-evaluating Theory of Mind evaluation in large language modelscited by 23
  14. 2024Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responsescited by 22
  15. 2024Evaluating Large Language Models with Psychometricscited by 22
  16. 2024LLM-Generated Black-box Explanations Can Be Adversarially Helpfulcited by 21
  17. 2024I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBenchcited by 20
  18. 2024The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Modelscited by 19
  19. 2025Large Language Models Do Not Simulate Human Psychologycited by 18
  20. 2023Susceptibility to Influence of Large Language Modelscited by 17
  21. 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
  22. 2023Reflective Linguistic Programming (RLP): A Stepping Stone in Socially-Aware AGI (SocialAGI)cited by 16
  23. 2024PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Modelscited by 16
  24. 2024Tracking the perspectives of interacting language modelscited by 14
  25. 2024Self-Cognition in Large Language Models: An Exploratory Studycited by 14
  26. 2023The System Model and the User Model: Exploring AI Dashboard Designcited by 12
  27. 2024RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Frameworkcited by 11
  28. 2025Synergizing Logical Reasoning, Knowledge Management and Collaboration in Multi-Agent LLM Systemcited by 10
  29. 2024Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verificationcited by 9
  30. 2024Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Modelscited by 9
  31. 2024Mind Your Theory: Theory of Mind Goes Deeper Than Reasoningcited by 9
  32. 2025From Prompts to Constructs: A Dual-Validity Framework for Large Language Model Research in Psychologycited by 9
  33. 2025Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Taskcited by 9
  34. 2026Learning to Simulate Human Dialoguecited by 9
  35. 2023Are NLP Models Good at Tracing Thoughts: An Overview of Narrative Understandingcited by 8
  36. 2025Bayesian Social Deduction with Graph-Informed Language Modelscited by 8
  37. 2025A validity-guided workflow for robust large language model research in psychologycited by 8
  38. 2025LVLMs are Bad at Overhearing Human Referential Communicationcited by 8

50 most cited of 218 All 218 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Boosting Theory-of-Mind Performance in Large Language Models via Prompting
  2. 2023Understanding Social Reasoning in Language Models with Language Models
  3. 2024Probing the Robustness of Theory of Mind in Large Language Models
  4. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
  5. 2024Multi-ToM: Evaluating Multilingual Theory of Mind Capabilities in Large Language Models
  6. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
  7. 2025Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decomposition
  8. 2025Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?