mlsanityData to 14 Oct 2026

Language Models use Lookbacks to Track Beliefs

arXiv 2505.14685 ↗First posted 20 May 2025cs.CLcited by 39

Not on the map. Why?

Abstract (arXiv)

How do language models (LMs) represent characters' beliefs, especially when those beliefs may differ from reality? This question lies at the heart of understanding the Theory of Mind (ToM) capabilities of LMs. We analyze LMs' ability to reason about characters' beliefs using causal mediation and abstraction. We construct a dataset, CausalToM, consisting of simple stories where two characters independently change the state of two objects, potentially unaware of each other's actions. Our investigation uncovers a pervasive algorithmic pattern that we call a lookback mechanism, which enables the LM to recall important information when it becomes necessary. The LM binds each character-object-state triple together by co-locating their reference information, represented as Ordering IDs (OIs), in low-rank subspaces of the state token's residual stream. When asked about a character's beliefs regarding the state of an object, the binding lookback retrieves the correct state OI and then the answer lookback retrieves the corresponding state token. When we introduce text specifying that one character is (not) visible to the other, we find that the LM first generates a visibility ID encoding the relation between the observing and the observed character OIs. In a visibility lookback, this ID is used to retrieve information about the observed character and update the observing character's beliefs. Our work provides insights into belief tracking mechanisms, taking a step toward reverse-engineering ToM reasoning in LMs.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

44 of 63 references.

References · 44

  1. 2023Qwen Technical Reportcited by 3,344
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2022In-context Learning and Induction Headscited by 964
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  10. 2021Causal Abstractions of Neural Networkscited by 422
  11. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  12. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
32 more
  1. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  2. 2021Implicit Representations of Meaning in Neural Language Modelscited by 180
  3. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  4. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
  5. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  6. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  7. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 149
  8. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  9. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 110
  10. 2023How do Language Models Bind Entities in Context?cited by 108
  11. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  12. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 89
  13. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 85
  14. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 76
  15. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  16. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  17. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 63
  18. 2024LLMs achieve adult human performance on higher-order theory of mind taskscited by 61
  19. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
  20. 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 51
  21. 2024NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internalscited by 45
  22. 2023Entity Tracking in Language Modelscited by 44
  23. 2024Monitoring Latent World States in Language Models with Propositional Probescited by 43
  24. 2023Discovering Variable Binding Circuitry with Desideratacited by 34
  25. 2024Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoningcited by 33
  26. 2024Language Models Represent Beliefs of Self and Otherscited by 31
  27. 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Modelscited by 31
  28. 2024Standards for Belief Representations in LLMscited by 30
  29. 2024Representational Analysis of Binding in Language Modelscited by 28
  30. 2024Mechanistic?cited by 25
  31. 2025Re-evaluating Theory of Mind evaluation in large language modelscited by 23
  32. 2024TimeToM: Temporal Space is the Key to Unlocking the Door of Large Language Models' Theory-of-Mindcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 30.

39 papers.

Sort:

By area:

  1. 2025Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Contextcited by 26
  2. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 26
  3. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
  4. 2025Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thoughtcited by 15
  5. 2025How Causal Abstraction Underpins Computational Explanationcited by 11
  6. 2026Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Conceptscited by 10
  7. 2025Mechanistic evaluation of Transformers and state space modelscited by 9
  8. 2025LLMs Process Lists With General Filter Headscited by 7
  9. 2026The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Researchcited by 6
  10. 2026Do Language Models Track Entities Across State Changes?cited by 6
  11. 2025In-Context Algebracited by 5
  12. 2026Mechanisms of Prompt-Induced Hallucination in Vision-Language Modelscited by 5
27 more
  1. 2026Do explanations generalize across large reasoning models?cited by 5
  2. 2026Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMscited by 5
  3. 2026Seeing to Generalize: How Visual Data Corrects Binding Shortcutscited by 4
  4. 2026The Dual Mechanisms of Spatial Variable Binding in Vision-Language Modelscited by 4
  5. 2026Decomposing Query-Key Feature Interactions Using Contrastive Covariancescited by 3
  6. 2026The Topological Trouble With Transformerscited by 3
  7. 2026Stories in Space: In-Context Learning Trajectories in Conceptual Belief Spacecited by 3
  8. 2026A retrieval conditioned rebinding circuit for dynamic entity tracking in large language modelscited by 3
  9. 2026HalluWorld: A Controlled Benchmark for Hallucination via Reference World Modelscited by 2
  10. 2026When Should Models Change Their Minds? Contextual Belief Management in Large Language Modelscited by 2
  11. 2026When Does Activation Steering Change What a Model Computes From?cited by 2
  12. 2026Compute Globally, Materialize Locally: The Memory Contract of Sparse Event-KVcited by 2
  13. 2025Artificial Phantasia: Emergent Mental Imagery in Large Language Modelscited by 1
  14. 2025Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updatescited by 1
  15. 2026PDDL-Mind: Large Language Models are Capable on Belief Reasoning with Reliable State Trackingcited by 1
  16. 2026Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMscited by 1
  17. 2025Circuit Distillationcited by 0
  18. 2026One mechanism for many mental spaces: a shared router over a value slot in language modelscited by 0
  19. 2026Belief-reality separation lives in routing over a shared value slot in language modelscited by 0
  20. 2026A Calibrated Test of Internal Action Maps: State Signals Without Global Affine Closurecited by 0
  21. 2026How Do Language Models Choose Between Context and Memory?cited by 0
  22. 2026Written as a Record, Read as an Address: What a Forward Pass Leaves in an Operation's KV Cachecited by 0
  23. 2026Lost with a Map: Conversational State and Behavioral Reliability in Language Modelscited by 0
  24. 2026Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes Itcited by 0
  25. 2026Shifting Mechanisms: How Positional Encoding Choice Shapes In-Context Retrievalcited by 0
  26. 2026The Geometry of Contextual Relations: Language Models Address Facts by Order of Mentioncited by 0
  27. 2026Beyond Belief Nesting: Rethinking Higher-Order Theory of Mind in Large Language Modelscited by 0