Relation Before Entity: Deferred Commitment in Language Model Factual Recall
Not on the map. Why?
Abstract (arXiv)
We ask whether relation-type information (e.g., capital-of) and entity-specific information (e.g., France to Paris) become causally active at the final-token position at the same depth during recall. Using four complementary causal diagnostics across four decoder-only models and eight prompt families, we find a robust temporal asymmetry: relation information becomes generation-controlling before entity information does. Relation onset precedes entity onset by 10-16 tested layers (31-44% of network depth) at threshold 0.4, with the ordering holding across all 16 model-threshold combinations for thresholds 0.2-0.5. Critically, entity information is not absent early: entity-token patching succeeds at 90-100% in early layers. Instead, entity commitment to generation is deferred: entity information is available at the entity-token position but becomes generation-controlling at the final token only after being routed there.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 12 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
8 of 12 references.
References · 8
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2022Mass-Editing Memory in a Transformercited by 1,001
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
- 2023Function Vectors in Large Language Modelscited by 308
- 2024How to use and interpret activation patchingcited by 214
- 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
- 2026Tracing Relational Knowledge Recall in Large Language Modelscited by 2
Built on by
Not cited yet.