# Do Language Models Track Entities Across State Changes? arXiv: https://arxiv.org/abs/2605.30233 ยท HTML: https://mlsanity.com/paper/2605.30233/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 44 of its 61 references that are in the map. - First posted on arXiv 28 May 2026. ## Builds on (most cited first, 30 of 44) - [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022) - [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022) - [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021) - [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023) - [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020) - [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019) - [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022) - [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021) - [Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking](https://mlsanity.com/paper/2403.09629/) (2024) - [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013) - [Chain of Thought Empowers Transformers to Solve Inherently Serial Problems](https://mlsanity.com/paper/2402.12875/) (2024) - [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023) - [AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders](https://mlsanity.com/paper/2501.17148/) (2025) - [Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations](https://mlsanity.com/paper/2303.02536/) (2023) - [Localizing Model Behavior with Path Patching](https://mlsanity.com/paper/2304.05969/) (2023) - [The Parallelism Tradeoff: Limitations of Log-Precision Transformers](https://mlsanity.com/paper/2207.00729/) (2022) - [The Illusion of State in State-Space Models](https://mlsanity.com/paper/2404.08819/) (2024) - [Implicit Representations of Meaning in Neural Language Models](https://mlsanity.com/paper/2106.00737/) (2021) - [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://mlsanity.com/paper/2311.01460/) (2023) - [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024) - [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024) - [What Formal Languages Can Transformers Express? A Survey](https://mlsanity.com/paper/2311.00208/) (2023) - [How do Language Models Bind Entities in Context?](https://mlsanity.com/paper/2310.17191/) (2023) - [Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information](https://mlsanity.com/paper/1808.08079/) (2018) - [Chess as a Testbed for Language Model State Tracking](https://mlsanity.com/paper/2102.13249/) (2021) - [Counterfactual Probabilities: Computational Methods, Bounds and Applications](https://mlsanity.com/paper/1302.6784/) (2013) - [Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models](https://mlsanity.com/paper/2403.15498/) (2024) - [Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction](https://mlsanity.com/paper/2105.06965/) (2021) - [Entity Tracking in Language Models](https://mlsanity.com/paper/2305.02363/) (2023) - [Monitoring Latent World States in Language Models with Propositional Probes](https://mlsanity.com/paper/2406.19501/) (2024) ## Built on by (most cited first, 6 of 6) - [A retrieval conditioned rebinding circuit for dynamic entity tracking in large language models](https://mlsanity.com/paper/2606.08644/) (2026) - [Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narratives](https://mlsanity.com/paper/2608.18083/) (2026) - [Written as a Record, Read as an Address: What a Forward Pass Leaves in an Operation's KV Cache](https://mlsanity.com/paper/2609.24635/) (2026) - [Positions Are Not Facts: The Mismatch Between KV Caches and Memory](https://mlsanity.com/paper/2609.33759/) (2026) - [Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It](https://mlsanity.com/paper/2609.36585/) (2026) - [When Context Changes: Understanding Update Failures in LLMs](https://mlsanity.com/paper/2609.38866/) (2026) ## Abstract (arXiv) Entity tracking (ET), the ability to keep track of states, is a fundamental skill that underlies complex reasoning. An increasing amount of work investigates how transformer language models (LMs) solve entity binding textitwithout state changes. However, there is limited understanding of how non-toy LMs address ET problems of realistic difficulties expressed in natural language. To this end, we investigate the mechanisms underlying ET in more complex scenarios featuring multiple state-changing operations. We find that LMs do not incrementally track world states across tokens or query-relevant states across layers, but simply aggregate relevant information in parallel at the last token when the query becomes evident. We further investigate mechanisms of individual operations (textttPUT, textttREMOVE, textttMOVE) to characterize this non-incremental ET mechanism. Surprisingly, LMs implement the textttREMOVE operation with a fragile global suppression tag; this global removal mechanism predicts various failure modes that we confirm behaviorally. We provide a mechanistic solution of nullifying this tag to partially address this issue. Overall, our findings reveal that LMs solve a fundamentally sequential task using a non-sequential strategy. More broadly, our work illustrates how behavioral and mechanistic analyses can fruitfully interact. Behavioral results inform mechanistic hypotheses, and insights from mechanistic analyses help build stronger behavioral evaluations by predicting failure modes missing from existing evaluations.