# Re-evaluating Theory of Mind evaluation in large language models

arXiv: https://arxiv.org/abs/2502.21098 · HTML: https://mlsanity.com/paper/2502.21098/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 141 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 30 of 43)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Data Contamination: From Memorization to Exploitation](https://mlsanity.com/paper/2203.08242/) (2022)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions](https://mlsanity.com/paper/2310.15421/) (2023)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks](https://mlsanity.com/paper/2305.10160/) (2023)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Circuit Component Reuse Across Tasks in Transformer Language Models](https://mlsanity.com/paper/2310.08744/) (2023)
- [Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker](https://mlsanity.com/paper/2306.00924/) (2023)
- [Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities](https://mlsanity.com/paper/2311.10227/) (2023)
- [MMToM-QA: Multimodal Theory of Mind Question Answering](https://mlsanity.com/paper/2401.08743/) (2024)
- [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://mlsanity.com/paper/2004.03066/) (2020)
- [How FaR Are Large Language Models From Agents with Theory-of-Mind?](https://mlsanity.com/paper/2310.03051/) (2023)
- [A fine-grained comparison of pragmatic language understanding in humans and language models](https://mlsanity.com/paper/2212.06801/) (2022)
- [Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4](https://mlsanity.com/paper/2309.17277/) (2023)
- [Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs](https://mlsanity.com/paper/2403.19827/) (2024)
- [Boosting Theory-of-Mind Performance in Large Language Models via Prompting](https://mlsanity.com/paper/2304.11490/) (2023)
- [HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models](https://mlsanity.com/paper/2310.16755/) (2023)
- [LLMs achieve adult human performance on higher-order theory of mind tasks](https://mlsanity.com/paper/2405.18870/) (2024)
- [Do Large Language Models know what humans know?](https://mlsanity.com/paper/2209.01515/) (2022)
- [PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities](https://mlsanity.com/paper/2401.07078/) (2024)
- [The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs](https://mlsanity.com/paper/2210.14986/) (2022)

## Built on by (most cited first, 23 of 23)

- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Language Models use Lookbacks to Track Beliefs](https://mlsanity.com/paper/2505.14685/) (2025)
- [Large Language Models Do Not Simulate Human Psychology](https://mlsanity.com/paper/2508.06950/) (2025)
- [Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task](https://mlsanity.com/paper/2507.16196/) (2025)
- [Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Fly](https://mlsanity.com/paper/2506.16755/) (2025)
- [The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind](https://mlsanity.com/paper/2506.20664/) (2025)
- [Metacognitive Sensitivity for Test-Time Dynamic Model Selection](https://mlsanity.com/paper/2512.10451/) (2025)
- [Simulated Annealing Enhances Theory-of-Mind Reasoning in Autoregressive Language Models](https://mlsanity.com/paper/2601.12269/) (2026)
- [Visuospatial Perspective Taking in Multimodal Language Models](https://mlsanity.com/paper/2603.23510/) (2026)
- [Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind](https://mlsanity.com/paper/2603.26089/) (2026)
- [Artificial Phantasia: Emergent Mental Imagery in Large Language Models](https://mlsanity.com/paper/2509.23108/) (2025)
- [Situated, Dynamic, and Subjective: Envisioning the Design of Theory-of-Mind-Enabled Everyday AI with Industry Practitioners](https://mlsanity.com/paper/2602.11342/) (2026)
- [Large Language Models Persuade Without Planning Theory of Mind](https://mlsanity.com/paper/2602.17045/) (2026)
- [Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action](https://mlsanity.com/paper/2606.31916/) (2026)
- [Metacognition in LLMs: Foundations, Progress, and Opportunities](https://mlsanity.com/paper/2607.11881/) (2026)
- [Assessing mentalization in humans and large language models](https://mlsanity.com/paper/2608.26291/) (2026)
- [Dynamic Theory of Mind as a Temporal Memory Problem: Evidence from Large Language Models](https://mlsanity.com/paper/2603.14646/) (2026)
- [Beneath the Surface: Investigating LLMs' Capabilities for Communicating with Subtext](https://mlsanity.com/paper/2604.05273/) (2026)
- [LEVANTE-bench: Multi-Scale Comparison of VLMs to Children Using Cognitive Tasks (or, "Is Your VLM Smarter Than a 5th Grader?")](https://mlsanity.com/paper/2606.05497/) (2026)
- [A Causal Model of Theory of Mind in Conflict for Artificial Intelligence](https://mlsanity.com/paper/2606.16944/) (2026)
- [Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetry](https://mlsanity.com/paper/2607.11053/) (2026)
- [Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning](https://mlsanity.com/paper/2608.04646/) (2026)
- [Using profiles of cognitive capability to assess AI suitability for workplace tasks](https://mlsanity.com/paper/2608.25623/) (2026)

## Abstract (arXiv)

The question of whether large language models (LLMs) possess Theory of Mind (ToM) -- often defined as the ability to reason about others' mental states -- has sparked significant scientific and public interest. However, the evidence as to whether LLMs possess ToM is mixed, and the recent growth in evaluations has not resulted in a convergence. Here, we take inspiration from cognitive science to re-evaluate the state of ToM evaluation in LLMs. We argue that a major reason for the disagreement on whether LLMs have ToM is a lack of clarity on whether models should be expected to match human behaviors, or the computations underlying those behaviors. We also highlight ways in which current evaluations may be deviating from "pure" measurements of ToM abilities, which also contributes to the confusion. We conclude by discussing several directions for future research, including the relationship between ToM and pragmatic communication, which could advance our understanding of artificial systems as well as human cognition.