# SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs

arXiv: https://arxiv.org/abs/2410.13648 · HTML: https://mlsanity.com/paper/2410.13648/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 68 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 30 of 32)

- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Understanding Social Reasoning in Language Models with Language Models](https://mlsanity.com/paper/2306.15448/) (2023)
- [FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions](https://mlsanity.com/paper/2310.15421/) (2023)
- [SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization](https://mlsanity.com/paper/2212.10465/) (2022)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker](https://mlsanity.com/paper/2306.00924/) (2023)
- [Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities](https://mlsanity.com/paper/2311.10227/) (2023)
- [MMToM-QA: Multimodal Theory of Mind Question Answering](https://mlsanity.com/paper/2401.08743/) (2024)
- [Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap](https://mlsanity.com/paper/2402.19450/) (2024)
- [How FaR Are Large Language Models From Agents with Theory-of-Mind?](https://mlsanity.com/paper/2310.03051/) (2023)
- [ToMBench: Benchmarking Theory of Mind in Large Language Models](https://mlsanity.com/paper/2402.15052/) (2024)
- [Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4](https://mlsanity.com/paper/2309.17277/) (2023)
- [Evaluating Theory of Mind in Question Answering](https://mlsanity.com/paper/1808.09352/) (2018)
- [HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models](https://mlsanity.com/paper/2310.16755/) (2023)
- [Do Large Language Models know what humans know?](https://mlsanity.com/paper/2209.01515/) (2022)
- [OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models](https://mlsanity.com/paper/2402.06044/) (2024)
- [Large Language Models Assume People are More Rational than We Really are](https://mlsanity.com/paper/2406.17055/) (2024)
- [MacGyver: Are Large Language Models Creative Problem Solvers?](https://mlsanity.com/paper/2311.09682/) (2023)
- [Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models](https://mlsanity.com/paper/2310.19619/) (2023)
- [InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context](https://mlsanity.com/paper/2406.12203/) (2024)
- [I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons](https://mlsanity.com/paper/2212.10060/) (2022)
- [Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models](https://mlsanity.com/paper/2407.06004/) (2024)
- [DREAM: Improving Situational QA by First Elaborating the Situation](https://mlsanity.com/paper/2112.08656/) (2021)
- [Do language models have coherent mental models of everyday things?](https://mlsanity.com/paper/2212.10029/) (2022)
- [AdvisorQA: Towards Helpful and Harmless Advice-seeking Question Answering with Collective Intelligence](https://mlsanity.com/paper/2404.11826/) (2024)
- [Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE](https://mlsanity.com/paper/2210.16407/) (2022)

## Built on by (most cited first, 30 of 57)

- [Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers](https://mlsanity.com/paper/2504.18412/) (2025)
- [Theory of Mind in Large Language Models: Assessment and Enhancement](https://mlsanity.com/paper/2505.00026/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues](https://mlsanity.com/paper/2502.21017/) (2025)
- [SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactions](https://mlsanity.com/paper/2506.23046/) (2025)
- [Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspective](https://mlsanity.com/paper/2504.10839/) (2025)
- [MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems](https://mlsanity.com/paper/2505.18943/) (2025)
- [Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests](https://mlsanity.com/paper/2502.14359/) (2025)
- [From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMs](https://mlsanity.com/paper/2502.17701/) (2025)
- [Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task](https://mlsanity.com/paper/2507.16196/) (2025)
- [Persuasion Should be Double-Blind: A Multi-Domain Dialogue Dataset With Faithfulness Based on Causal Theory of Mind](https://mlsanity.com/paper/2502.21297/) (2025)
- [TreeSynth: Synthesizing Diverse Data from Scratch via Tree-Guided Subspace Partitioning](https://mlsanity.com/paper/2503.17195/) (2025)
- [One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligence](https://mlsanity.com/paper/2602.03109/) (2026)
- [R^3-VQA: "Read the Room" by Video Social Reasoning](https://mlsanity.com/paper/2505.04147/) (2025)
- [PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting](https://mlsanity.com/paper/2606.08878/) (2026)
- [The Thin Line Between Comprehension and Persuasion in LLMs](https://mlsanity.com/paper/2507.01936/) (2025)
- [ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions](https://mlsanity.com/paper/2509.05066/) (2025)
- [TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?](https://mlsanity.com/paper/2509.17054/) (2025)
- [Social-R1: Towards Human-like Social Reasoning in LLMs](https://mlsanity.com/paper/2603.09249/) (2026)
- [CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models](https://mlsanity.com/paper/2601.15628/) (2026)
- [Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs](https://mlsanity.com/paper/2602.16085/) (2026)
- [EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents](https://mlsanity.com/paper/2605.09826/) (2026)
- [Playpen: An Environment for Exploring Learning Through Conversational Interaction](https://mlsanity.com/paper/2504.08590/) (2025)
- [Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind](https://mlsanity.com/paper/2603.26089/) (2026)
- [DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories](https://mlsanity.com/paper/2604.20443/) (2026)
- [They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On It](https://mlsanity.com/paper/2607.03598/) (2026)
- [Towards Safety Evaluations of Theory of Mind in Large Language Models](https://mlsanity.com/paper/2506.17352/) (2025)
- [On Emergent Social World Models -- Evidence for Functional Integration of Theory of Mind and Pragmatic Reasoning in Language Models](https://mlsanity.com/paper/2602.10298/) (2026)
- [Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind](https://mlsanity.com/paper/2604.11666/) (2026)
- [A formal definition and meta-model for a machine theory of mind](https://mlsanity.com/paper/2606.03471/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly tested for a "Theory of Mind" (ToM) - the ability to attribute mental states to oneself and others. Yet most evaluations stop at explicit belief attribution in classical toy stories or stylized tasks, leaving open the questions of whether LLMs can implicitly apply such knowledge to predict human behavior, or to judge an observed behavior, in diverse scenarios. We introduce SimpleToM, a benchmark that advances ToM evaluation along two novel axes. First, it probes multiple levels of ToM reasoning, from mental state inference (explicit ToM) to behavior prediction and judgment (applied ToM). Second, it situates these tasks in diverse, everyday scenarios - such as supermarkets, hospitals, schools, and offices - where information asymmetries naturally arise (e.g., hidden defects in grocery store items, incomplete information in provider-patient interactions, or restricted access to locked devices). SimpleToM contains concise stories (e.g., "The can of Pringles has moldy chips in it. Mary picks up the can in the supermarket and walks to the cashier."), each with three questions that test different degrees of ToM reasoning, asking models to predict: (a) mental states ("Is Mary aware of the mold?"), (b) behaviors ("Will Mary pay for the chips or report the mold?"), and (c) judgments ("Mary paid for the chips. Was that reasonable?"). Experiments reveal a striking gap: state-of-the-art models often reliably infer mental state (a), but fail at applying knowledge about the mental state for secondary predictions, with performance dropping sharply for behavior prediction (b) and further for behavior judgment (c). This exposes a critical fragility in LLMs' social reasoning in terms of what they know (explicit ToM) versus how well they can implicitly apply that knowledge for predictions (applied ToM).