mlsanityData to 14 Oct 2026

One mechanism for many mental spaces: a shared router over a value slot in language models

arXiv 2607.10248 ↗First posted 11 Jul 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Language builds discourse contexts other than the actual: a painting, a belief, a memory, a hypothetical. Each is a mental space in which the same entity can take a different value, as when a flower is red in reality but purple in a portrait. Formal semantics keeps these contexts apart because their logics differ (modal, temporal, doxastic, depictive). Fauconnier's mental-space theory, by contrast, treats them as one space-building operation. We ask which of these a transformer language model implements, and find a mechanistic version of Fauconnier's unification. The model uses one router/slot format across the inventory: a reusable value slot stores attributed content, and a causally manipulable router (the space index) selects which space is read. A subspace trained with Distributed Alignment Search to control one space type, counterfactual, belief, fictional, or temporal, also controls the others, well above a random floor, on three model families. Belief, which formal semantics marks as a distinct case, is not specially separated. The router is low-rank, composes additively with entity identity, and acts through a few late-layer heads. Two further results show the mechanism drives inference and composes: a subspace trained on a rule-derived conclusion flips what the model infers while dissociating from what it reports, and composing space-builders mints a fresh router over the shared slot. This paper establishes the cross-type generality. A companion paper develops belief in depth, because of its special status in philosophy, psychology, and linguistics (epistemology, theory of mind, and propositional attitude reports).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 22 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

13 of 22 references.

References · 13

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  2. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  3. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 191
  4. 2021Implicit Representations of Meaning in Neural Language Modelscited by 180
  5. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  6. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 62
  7. 2025Language Models use Lookbacks to Track Beliefscited by 39
  8. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 37
  9. 2024Representational Analysis of Binding in Language Modelscited by 28
  10. 2023Counterfactual reasoning: Testing language models' understanding of hypothetical scenarioscited by 27
  11. 2025How Do Transformers Learn Variable Binding in Symbolic Programs?cited by 21
  12. 2025Just-in-time and distributed task representations in language modelscited by 9
1 more
  1. 2025Counterfactual reasoning: an analysis of in-context emergencecited by 4

Built on by

Not cited yet.