mlsanityData to 14 Oct 2026

Enhancing Conversational Agents with Theory of Mind: Aligning Beliefs, Desires, and Intentions for Human-Like Interaction

arXiv 2502.14171 ↗First posted 20 Feb 2025cs.CLcited by 3

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Natural language interaction with agentic Artificial Intelligence (AI), driven by Large Language Models (LLMs), is expected to remain a dominant paradigm in the near future. While humans instinctively align their communication with mental states -- an ability known as Theory of Mind (ToM), current LLM powered systems exhibit significant limitations in this regard. This study examines the extent to which open source language models (LLaMA) can capture and preserve ToM related information and how effectively it contributes to consistent ToM reasoning in generated responses. We further investigate whether explicit manipulation of ToM related components, such as beliefs, desires, and intentions, can enhance response alignment. Experiments on two LLaMA 3 variants demonstrate that incorporating ToM informed alignment improves response quality, achieving win rates of 67 and 63 percent for the 3B and 8B models, respectively. These findings highlight the potential of ToM driven strategies to improve alignment in LLM based conversational agents.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 33 of its 76 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

33 of 76 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  4. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  5. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  6. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  7. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  8. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  9. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
  10. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  11. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
  12. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
21 more
  1. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 162
  2. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  3. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  4. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 77
  5. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 67
  6. 2021CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systemscited by 62
  7. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 62
  8. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
  9. 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 51
  10. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 51
  11. 2023Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Testscited by 48
  12. 2024Designing a Dashboard for Transparency and Control of Conversational AIcited by 46
  13. 2024Large Language Models Produce Responses Perceived to be Empathiccited by 36
  14. 2024Backward Lens: Projecting Language Model Gradients into the Vocabulary Spacecited by 32
  15. 2022Let's Negotiate! A Survey of Negotiation Dialogue Systemscited by 31
  16. 2024Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responsescited by 22
  17. 2024Analysing the Residual Stream of Language Models Under Knowledge Conflictscited by 17
  18. 2025Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decompositioncited by 17
  19. 2021A Survey on AI Assurancecited by 16
  20. 2023Structured World Representations in Maze-Solving Transformerscited by 16
  21. 2024A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniquescited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025The Effect of Belief Boxes and Open-mindedness on Persuasioncited by 3
  2. 2025A Survey of the State-of-the-Art in Conversational Question Answering Systemscited by 1
  3. 2026Mechanistic Indicators of Steering Effectiveness in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Infusing Theory of Mind into Socially Intelligent LLM Agents
  2. 2025Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflection
  3. 2026Adaptive Theory of Mind for LLM-based Multi-Agent Coordination
  4. 2026Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action
  5. 2024LLM Theory of Mind and Alignment: Opportunities and Risks
  6. 2024Probing the Robustness of Theory of Mind in Large Language Models
  7. 2026Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
  8. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation