mlsanityData to 7 Oct 2026

How FaR Are Large Language Models From Agents with Theory-of-Mind?

arXiv 2310.03051 ↗First posted 4 Oct 2023cs.CL cs.AIcited by 75

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

"Thinking is for Doing." Humans can infer other people's mental states from observations--an ability called Theory-of-Mind (ToM)--and subsequently act pragmatically on those inferences. Existing question answering benchmarks such as ToMi ask models questions to make inferences about beliefs of characters in a story, but do not test whether models can then use these inferences to guide their actions. We propose a new evaluation paradigm for large language models (LLMs): Thinking for Doing (T4D), which requires models to connect inferences about others' mental states to actions in social scenarios. Experiments on T4D demonstrate that LLMs such as GPT-4 and PaLM 2 seemingly excel at tracking characters' beliefs in stories, but they struggle to translate this capability into strategic action. Our analysis reveals the core challenge for LLMs lies in identifying the implicit inferences about mental states without being explicitly asked about as in ToMi, that lead to choosing the correct action in T4D. To bridge this gap, we introduce a zero-shot prompting framework, Foresee and Reflect (FaR), which provides a reasoning structure that encourages LLMs to anticipate future challenges and reason about potential actions. FaR boosts GPT-4's performance from 50% to 71% on T4D, outperforming other prompting methods such as Chain-of-Thought and Self-Ask. Moreover, FaR generalizes to diverse out-of-distribution story structures and scenarios that also require ToM inferences to choose an action, consistently outperforming other methods including few-shot in-context learning.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 24 of its 52 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

24 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language Models as Agent Models3 Dec

Other references · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  10. 2023PaLM 2 Technical Reportcited by 1,154
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  12. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
11 more
  1. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  2. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  3. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  4. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  5. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  6. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  7. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  8. 2018Evaluating Theory of Mind in Question Answeringcited by 64
  9. 2021MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Taskscited by 62
  10. 2022Do Large Language Models know what humans know?cited by 60
  11. 2022I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragonscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 35; 2025: 18; 2026: 16.

75 papers.

Sort:

By area:

  1. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  2. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  3. 2024Self-Discover: Large Language Models Self-Compose Reasoning Structurescited by 108
  4. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  5. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  6. 2024ToMBench: Benchmarking Theory of Mind in Large Language Modelscited by 70
  7. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
  8. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 51
  9. 2023Merlin:Empowering Multimodal LLMs with Foresight Mindscited by 46
  10. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 42
  11. 2025Language Models use Lookbacks to Track Beliefscited by 36
  12. 2024Agents Thinking Fast and Slow: A Talker-Reasoner Architecturecited by 35
38 more
  1. 2023Towards A Holistic Landscape of Situated Theory of Mind in Large Language Modelscited by 32
  2. 2024Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoningcited by 30
  3. 2024Language Models Represent Beliefs of Self and Otherscited by 29
  4. 2024Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?cited by 28
  5. 2023Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Gamecited by 27
  6. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  7. 2025Theory of Mind in Large Language Models: Assessment and Enhancementcited by 24
  8. 2025Re-evaluating Theory of Mind evaluation in large language modelscited by 23
  9. 2024Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Modelscited by 22
  10. 2024Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulationcited by 21
  11. 2026Large Language Model Reasoning Failurescited by 20
  12. 2024A Notion of Complexity for Theory of Mind via Discrete World Modelscited by 16
  13. 2024Perceptions of Linguistic Uncertainty by Language Models and Humanscited by 15
  14. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 14
  15. 2024Evolving AI Collectives to Enhance Human Diversity and Enable Self-Regulationcited by 13
  16. 2025From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMscited by 12
  17. 2024Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Datacited by 10
  18. 2024PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Gamescited by 10
  19. 2025Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflectioncited by 10
  20. 2024Recent Advancement of Emotion Cognition in Large Language Modelscited by 9
  21. 2024Friends-MMC: A Dataset for Multi-modal Multi-party Conversation Understandingcited by 9
  22. 2025Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Taskcited by 9
  23. 2024Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Modelscited by 8
  24. 2025LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Gamecited by 8
  25. 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
  26. 2025HARBOR: Exploring Persona Dynamics in Multi-Agent Competitioncited by 6
  27. 2025GenoMAS: A Multi-Agent Framework for Scientific Discovery via Code-Driven Gene Expression Analysiscited by 6
  28. 2024CHARTOM: A Visual Theory-of-Mind Benchmark for LLMs on Misleading Chartscited by 5
  29. 2024Belief in the Machine: Investigating Epistemological Blind Spots of Language Modelscited by 5
  30. 2024Hermes: A Large Language Model Framework on the Journey to Autonomous Networkscited by 5
  31. 2024MindForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learningcited by 5
  32. 2024Multi-ToM: Evaluating Multilingual Theory of Mind Capabilities in Large Language Modelscited by 5
  33. 2025From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Modelscited by 5
  34. 2025ToM-SSI: Evaluating Theory of Mind in Situated Social Interactionscited by 5
  35. 2024Improving Language Models for Emotion Analysis: Insights from Cognitive Sciencecited by 4
  36. 2024Auto-Evolve: Enhancing Large Language Model's Performance via Self-Reasoning Frameworkcited by 4
  37. 2025The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mindcited by 4
  38. 2025CaughtCheating: Is Your MLLM a Good Cheating Detective? Exploring the Boundary of Visual Perception and Reasoningcited by 4

50 most cited of 75 All 75 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Understanding Social Reasoning in Language Models with Language Models
  2. 2025Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decomposition
  3. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities
  4. 2026Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
  5. 2025Language Models Might Not Understand You: Evaluating Theory of Mind via Story Prompting
  6. 2024Position: Theory of Mind Benchmarks are Broken for Large Language Models
  7. 2025Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?
  8. 2025The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characters