mlsanityData to 7 Oct 2026

ToMBench: Benchmarking Theory of Mind in Large Language Models

arXiv 2402.15052 ↗First posted 23 Feb 2024cs.CL cs.AIcited by 70

Not on the map. Why?

Abstract (arXiv)

Theory of Mind (ToM) is the cognitive capability to perceive and ascribe mental states to oneself and others. Recent research has sparked a debate over whether large language models (LLMs) exhibit a form of ToM. However, existing ToM evaluations are hindered by challenges such as constrained scope, subjective judgment, and unintended contamination, yielding inadequate assessments. To address this gap, we introduce ToMBench with three key characteristics: a systematic evaluation framework encompassing 8 tasks and 31 abilities in social cognition, a multiple-choice question format to support automated and unbiased evaluation, and a build-from-scratch bilingual inventory to strictly avoid data leakage. Based on ToMBench, we conduct extensive experiments to evaluate the ToM performance of 10 popular LLMs across tasks and abilities. We find that even the most advanced LLMs like GPT-4 lag behind human performance by over 10% points, indicating that LLMs have not achieved a human-level theory of mind yet. Our aim with ToMBench is to enable an efficient and effective evaluation of LLMs' ToM capabilities, thereby facilitating the development of LLMs with inherent social intelligence.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Builds on 15 of its 74 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

15 of 74 references.

References · 15

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  6. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  7. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  8. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  9. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  10. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
  11. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 86
  12. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
3 more
  1. 2023Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Testscited by 46
  2. 2022Annotation Error Detection: Analyzing the Past and Present for a More Coherent Futurecited by 45
  3. 2023ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mindcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 31; 2026: 34.

70 papers.

Sort:

By area:

  1. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 51
  2. 2024Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoningcited by 30
  3. 2025Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Modelscited by 28
  4. 2025On Benchmarking Human-Like Intelligence in Machinescited by 24
  5. 2026CooperBench: Why Coding Agents Cannot be Your Teammates Yetcited by 20
  6. 2024A Notion of Complexity for Theory of Mind via Discrete World Modelscited by 16
  7. 2025SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactionscited by 16
  8. 2025NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learningcited by 15
  9. 2025Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspectivecited by 14
  10. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systemscited by 12
  11. 2025ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mindcited by 12
  12. 2025Persuasion Should be Double-Blind: A Multi-Domain Dialogue Dataset With Faithfulness Based on Causal Theory of Mindcited by 8
38 more
  1. 2025LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Gamecited by 8
  2. 2026Unveiling the Cognitive Compass: Theory-of-Mind-Guided Multimodal Emotion Reasoningcited by 8
  3. 2025Infusing Theory of Mind into Socially Intelligent LLM Agentscited by 7
  4. 2025Are Large Language Models Sensitive to the Motives Behind Communication?cited by 7
  5. 2024CHARTOM: A Visual Theory-of-Mind Benchmark for LLMs on Misleading Chartscited by 5
  6. 2025ToM-SSI: Evaluating Theory of Mind in Situated Social Interactionscited by 5
  7. 2026HumanLLM: Towards Personalized Understanding and Simulation of Human Naturecited by 5
  8. 2026AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnessescited by 5
  9. 2025MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?cited by 4
  10. 2025The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mindcited by 4
  11. 2025Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Frameworkcited by 4
  12. 2026Social-R1: Towards Human-like Social Reasoning in LLMscited by 4
  13. 2026JevOut: Natural Context Can Flip Decision Modelscited by 4
  14. 2025Towards properly implementing Theory of Mind in AI systems: An account of four misconceptionscited by 3
  15. 2025On the Adaptive Psychological Persuasion of Large Language Modelscited by 3
  16. 2025H2HTalk: Evaluating Large Language Models as Emotional Companioncited by 3
  17. 2026EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Managementcited by 3
  18. 2024RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreakingcited by 2
  19. 2025SNS-Bench-VL: Benchmarking Multimodal Large Language Models in Social Networking Servicescited by 2
  20. 2025Think Socially via Cognitive Reasoningcited by 2
  21. 2025HugAgent: A Human Simulation Benchmark for Individual-Level Reasoningcited by 2
  22. 2026Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mindcited by 2
  23. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  24. 2025CLAIM: An Intent-Driven Multi-Agent Framework for Analyzing Manipulation in Courtroom Dialoguescited by 1
  25. 2025Are Vision Language Models Cross-Cultural Theory of Mind Reasoners?cited by 1
  26. 2026Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mindcited by 1
  27. 2026OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modelingcited by 1
  28. 2026Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Modelscited by 1
  29. 2026Mental World Modelingcited by 1
  30. 2026SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Designcited by 1
  31. 2025Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Modelscited by 0
  32. 2025textttDIAMONDs: A Dataset for mathbbDynamic mathbbInformation mathbbAnd mathbbMental modeling mathbbOf mathbbNumeric mathbbDiscussionscited by 0
  33. 2025TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning for Enhancing LLMs' Social Intelligencecited by 0
  34. 2025Language Models Might Not Understand You: Evaluating Theory of Mind via Story Promptingcited by 0
  35. 2025Growing Perspectives: Modelling Embodied Perspective Taking and Inner Narrative Development Using Large Language Modelscited by 0
  36. 2025When Researchers Say Mental Model/Theory of Mind of AI, What Are They Really Talking About?cited by 0
  37. 2025Single-Agent Scaling Fails Multi-Agent Intelligence: Towards Foundation Models with Native Multi-Agent Intelligencecited by 0
  38. 2025Observer, Not Player: Simulating Theory of Mind in LLMs through Game Observationcited by 0

50 most cited of 70 All 70 (JSON)