mlsanityData to 7 Oct 2026

Evaluating Theory of Mind in Question Answering

arXiv 1808.09352 ↗First posted 28 Aug 2018cs.CLcited by 64

Not on the map. Why?

Abstract (arXiv)

We propose a new dataset for evaluating question answering models with respect to their capacity to reason about beliefs. Our tasks are inspired by theory-of-mind experiments that examine whether children are able to reason about the beliefs of others, in particular when those beliefs differ from reality. We evaluate a number of recent neural models with memory augmentation. We find that all fail on our tasks, which require keeping track of inconsistent states of the world; moreover, the models' accuracy decreases notably when random sentences are introduced to the tasks at test.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 21; 2024: 16; 2025: 10; 2026: 7.

64 papers.

Sort:

By area:

  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  3. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  4. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  5. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  6. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  7. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  8. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  9. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  10. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 84
  11. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
  12. 2022A fine-grained comparison of pragmatic language understanding in humans and language modelscited by 69
38 more
  1. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 61
  2. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
  3. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 51
  4. 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 49
  5. 2023Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Testscited by 46
  6. 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
  7. 2023Towards A Holistic Landscape of Situated Theory of Mind in Large Language Modelscited by 32
  8. 2024Language Models Represent Beliefs of Self and Otherscited by 29
  9. 2023ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mindcited by 28
  10. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  11. 2025Theory of Mind in Large Language Models: Assessment and Enhancementcited by 24
  12. 2021Cognitive science as a source of forward and inverse models of human decisions for robotics and controlcited by 22
  13. 2022I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragonscited by 22
  14. 2024I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBenchcited by 20
  15. 2024TimeToM: Temporal Space is the Key to Unlocking the Door of Large Language Models' Theory-of-Mindcited by 20
  16. 2024Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Modelscited by 20
  17. 2022Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mindcited by 19
  18. 2023MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logiccited by 18
  19. 2021Learning Triadic Belief Dynamics in Nonverbal Communication from Videoscited by 17
  20. 2022ALERT: Adapting Language Models to Reasoning Taskscited by 17
  21. 2025Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decompositioncited by 17
  22. 2025Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal Evolution of Human Statescited by 17
  23. 2023Computational Language Acquisition with Theory of Mindcited by 15
  24. 2023Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalizationcited by 15
  25. 2023QASiNa: Religious Domain Question Answering using Sirah Nabawiyahcited by 14
  26. 2024Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Groundcited by 10
  27. 2023A Review on Machine Theory of Mindcited by 9
  28. 2023Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialoguecited by 9
  29. 2024Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Modelscited by 9
  30. 2025The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characterscited by 8
  31. 2022Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Modelscited by 7
  32. 2024Are LLMs good pragmatic speakers?cited by 7
  33. 2023Memory-Augmented Theory of Mind Networkcited by 6
  34. 2024Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Studycited by 6
  35. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  36. 2024BDIQA: A New Dataset for Video Question Answering to Explore Cognitive Reasoning through Theory of Mindcited by 5
  37. 2025TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?cited by 5
  38. 2026Reinforcing Human Behavior Simulation via Verbal Feedbackcited by 4

50 most cited of 64 All 64 (JSON)