mlsanityData to 7 Oct 2026

Boosting Theory-of-Mind Performance in Large Language Models via Prompting

arXiv 2304.11490 ↗First posted 22 Apr 2023cs.AI cs.CLcited by 64

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

Large language models (LLMs) excel in many tasks in 2023, but they still face challenges in complex reasoning. Theory-of-mind (ToM) tasks, which require understanding agents' beliefs, goals, and mental states, are essential for common-sense reasoning involving humans, making it crucial to enhance LLM performance in this area. This study measures the ToM performance of GPT-4 and three GPT-3.5 variants (Davinci-2, Davinci-3, GPT-3.5-Turbo), and investigates the effectiveness of in-context learning in improving their ToM comprehension. We evaluated prompts featuring two-shot chain of thought reasoning and step-by-step thinking instructions. We found that LLMs trained with Reinforcement Learning from Human Feedback (RLHF) (all models excluding Davinci-2) improved their ToM accuracy via in-context learning. GPT-4 performed best in zero-shot settings, reaching nearly 80% ToM accuracy, but still fell short of the 87% human accuracy on the test set. However, when supplied with prompts for in-context learning, all RLHF-trained LLMs exceeded 80% ToM accuracy, with GPT-4 reaching 100%. These results demonstrate that appropriate prompting enhances LLM ToM reasoning, and they underscore the context-dependent nature of LLM cognitive capacities.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 72 references that are in the map. First posted on arXiv 22 Apr 2023.

Builds on

15 of 72 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  9. 2022Teaching Small Language Models to Reasoncited by 279
  10. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  11. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  12. 2023Why think step by step? Reasoning emerges from the locality of experiencecited by 103
3 more
  1. 2023A Theory of Emergent In-Context Learning as Implicit Structure Inductioncited by 86
  2. 2022Do Large Language Models know what humans know?cited by 60
  3. 2022Large Language Models and the Reverse Turing Testcited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 29; 2024: 16; 2025: 13; 2026: 6.

64 papers.

Sort:

By area:

  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  2. 2023Playing repeated games with Large Language Modelscited by 213
  3. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  4. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  5. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 129
  6. 2023Deception Abilities Emerged in Large Language Modelscited by 99
  7. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 84
  8. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  9. 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
  10. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 61
  11. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59
  12. 2023Emotional Intelligence of Large Language Modelscited by 58
38 more
  1. 2023Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Testscited by 46
  2. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  3. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  4. 2023Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Modelscited by 36
  5. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  6. 2025Language Models use Lookbacks to Track Beliefscited by 36
  7. 2023Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratingscited by 33
  8. 2023Towards A Holistic Landscape of Situated Theory of Mind in Large Language Modelscited by 32
  9. 2023Probing the Creativity of Large Language Models: Can models produce divergent semantic association?cited by 31
  10. 2024Language Models Represent Beliefs of Self and Otherscited by 29
  11. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  12. 2025Re-evaluating Theory of Mind evaluation in large language modelscited by 23
  13. 2025Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemmacited by 22
  14. 2024Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulationcited by 21
  15. 2023Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understandingcited by 20
  16. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Informationcited by 20
  17. 2023MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logiccited by 18
  18. 2023An evolutionary model of personality traits related to cooperative behavior using a large language modelcited by 18
  19. 2024LLM-3D Print: Large Language Models To Monitor and Control 3D Printingcited by 18
  20. 2025Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decompositioncited by 17
  21. 2023Does ChatGPT have Theory of Mind?cited by 16
  22. 2024A Notion of Complexity for Theory of Mind via Discrete World Modelscited by 16
  23. 2024Detecting Conversational Mental Manipulation with Intent-Aware Promptingcited by 16
  24. 2023Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalizationcited by 15
  25. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 14
  26. 2023What Should Data Science Education Do with Large Language Models?cited by 13
  27. 2023Evaluation of large language models for discovery of gene set functioncited by 13
  28. 2025Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflectioncited by 10
  29. 2023Unpacking the Ethical Value Alignment in Big Modelscited by 9
  30. 2024Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Modelscited by 8
  31. 2023OntoChatGPT Information System: Ontology-Driven Structured Prompts for ChatGPT Meta-Learningcited by 6
  32. 2024CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Modelscited by 6
  33. 2025Towards Personalized Conversational Sales Agents: Contextual User Profiling for Strategic Actioncited by 6
  34. 2025Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Flycited by 6
  35. 2025ToM-SSI: Evaluating Theory of Mind in Situated Social Interactionscited by 5
  36. 2023Violation of Expectation via Metacognitive Prompting Reduces Theory of Mind Prediction Error in Large Language Modelscited by 4
  37. 2024Teaching Machines to Code: Smart Contract Translation with LLMscited by 3
  38. 2025Towards properly implementing Theory of Mind in AI systems: An account of four misconceptionscited by 3

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?
  2. 2023Evaluating Large Language Models in Theory of Mind Tasks
  3. 2026Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
  4. 2026Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
  5. 2024Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models
  6. 2024Probing the Robustness of Theory of Mind in Large Language Models
  7. 2026Reasoning Promotes Robustness in Theory of Mind Tasks
  8. 2026To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Tasks