mlsanityData to 14 Oct 2026

Relation Before Entity: Deferred Commitment in Language Model Factual Recall

arXiv 2609.17537 ↗First posted 13 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We ask whether relation-type information (e.g., capital-of) and entity-specific information (e.g., France to Paris) become causally active at the final-token position at the same depth during recall. Using four complementary causal diagnostics across four decoder-only models and eight prompt families, we find a robust temporal asymmetry: relation information becomes generation-controlling before entity information does. Relation onset precedes entity onset by 10-16 tested layers (31-44% of network depth) at threshold 0.4, with the ordering holding across all 16 model-threshold combinations for thresholds 0.2-0.5. Critically, entity information is not absent early: entity-token patching succeeds at 90-100% in early layers. Instead, entity commitment to generation is deferred: entity information is available at the entity-token position but becomes generation-controlling at the final token only after being routed there.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 12 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

8 of 12 references.

References · 8

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  3. 2022Mass-Editing Memory in a Transformercited by 1,001
  4. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  5. 2023Function Vectors in Large Language Modelscited by 308
  6. 2024How to use and interpret activation patchingcited by 214
  7. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  8. 2026Tracing Relational Knowledge Recall in Large Language Modelscited by 2

Built on by

Not cited yet.