mlsanityData to 7 Oct 2026

Language Models Represent Beliefs of Self and Others

arXiv 2402.18496 ↗First posted 28 Feb 2024cs.AI cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

Understanding and attributing mental states, known as Theory of Mind (ToM), emerges as a fundamental capability for human social reasoning. While Large Language Models (LLMs) appear to possess certain ToM abilities, the mechanisms underlying these capabilities remain elusive. In this study, we discover that it is possible to linearly decode the belief status from the perspectives of various agents through neural activations of language models, indicating the existence of internal representations of self and others' beliefs. By manipulating these representations, we observe dramatic changes in the models' ToM performance, underscoring their pivotal role in the social reasoning process. Additionally, our findings extend to diverse social reasoning tasks that involve different causal inference patterns, suggesting the potential generalizability of these representations.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 37 of its 66 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

37 of 66 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2023Mistral 7Bcited by 3,065
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2024Mixtral of Expertscited by 1,785
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  9. 2018Neural Network Acceptability Judgmentscited by 952
  10. 2022Toy Models of Superpositioncited by 928
  11. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  12. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
25 more
  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  2. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  3. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  4. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  5. 2023Language Models Represent Space and Timecited by 306
  6. 2023Function Vectors in Large Language Modelscited by 292
  7. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  8. 2023AI Alignment: A Comprehensive Surveycited by 256
  9. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  10. 2018Machine Theory of Mindcited by 222
  11. 2022Relative representations enable zero-shot latent space communicationcited by 175
  12. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  13. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  14. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 86
  15. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
  16. 2018Evaluating Theory of Mind in Question Answeringcited by 64
  17. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  18. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  19. 2021AGENT: A Benchmark for Core Psychological Reasoningcited by 42
  20. 2019When Choosing Plausible Alternatives, Clever Hans can be Clevercited by 39
  21. 2021Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of otherscited by 39
  22. 2019Theory of Minds: Understanding Behavior in Groups Through Inverse Planningcited by 30
  23. 2023ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mindcited by 28
  24. 2024Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion?cited by 22
  25. 2023Social Motion Prediction with Cognitive Hierarchiescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 17.

29 papers.

Sort:

By area:

  1. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  2. 2025Language Models use Lookbacks to Track Beliefscited by 36
  3. 2025Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steeringcited by 24
  4. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  5. 2025The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hackingcited by 18
  6. 2024Towards Safe and Honest AI Agents with Neural Self-Other Overlapcited by 6
  7. 2026Position: Anthropomorphic Misalignment Research Needs Stronger Evidencecited by 6
  8. 2025The Yokai Learning Environment: Tracking Beliefs Over Space and Timecited by 3
  9. 2026Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancementcited by 3
  10. 2026When Role-playing, Do Models Believe What They Say?cited by 3
  11. 2026Beyond representational alignment with brain-guided language models for robust reasoningcited by 3
  12. 2025Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Marketscited by 2
17 more
  1. 2026Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mindcited by 2
  2. 2024DynFrs: An Efficient Framework for Machine Unlearning in Random Forestcited by 1
  3. 2025Modeling the Mental World for Embodied AI: A Comprehensive Reviewcited by 1
  4. 2026Large language models accurately predict public perceptions of support for climate action worldwidecited by 1
  5. 2026On Emergent Social World Models -- Evidence for Functional Integration of Theory of Mind and Pragmatic Reasoning in Language Modelscited by 1
  6. 2025Circuit Distillationcited by 0
  7. 2025When Researchers Say Mental Model/Theory of Mind of AI, What Are They Really Talking About?cited by 0
  8. 2026Neural Synchrony Between Socially Interacting Language Modelscited by 0
  9. 2026Probing the Limits of the Lie Detector Approach to LLM Deceptioncited by 0
  10. 2026Do Large Language Models Get Caught in Hofstadter-Mobius Loops?cited by 0
  11. 2026Video-Only ToM: Enhancing Theory of Mind in Multimodal Large Language Modelscited by 0
  12. 2026Belief-Aware VLM Model for Human-like Reasoningcited by 0
  13. 2026OSCToM: RL-Guided Adversarial Generation for High-Order Theory of Mindcited by 0
  14. 2026User-Aware Active Knowledge Acquisition for Emotional Support Dialoguecited by 0
  15. 2026From Economic Agents to Agentic Economies: A Systems Blueprint for Economic World Modelscited by 0
  16. 2026When Users Change Their Minds: Measuring and Repairing Intent Drift in LLM Agentscited by 0
  17. 2026When Models Don't Manipulate Manifolds: The Geometry of a Comparison Taskcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models
  2. 2026Dynamic Theory of Mind as a Temporal Memory Problem: Evidence from Large Language Models
  3. 2024Multi-ToM: Evaluating Multilingual Theory of Mind Capabilities in Large Language Models
  4. 2024Probing the Robustness of Theory of Mind in Large Language Models
  5. 2025Doing Things with Words: Rethinking Theory of Mind Simulation in Large Language Models
  6. 2026Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs
  7. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs
  8. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems