A Calibrated Test of Internal Action Maps: State Signals Without Global Affine Closure
Not on the map. Why?
Abstract (arXiv)
A hidden state signal can be decodable or causally usable without supporting a reusable action map. We test whether action maps fitted without a source reach its natural post-action activation and compose. We organize the tests as an evidence lattice and validate the geometric branch on a known affine S_5 carrier: all held-source folds pass one-step, composition, inverse, decoding, and commutativity gates. Structured curvature and held-domain conjugacy raise error monotonically, but only 23/30 strongest cells flip a closure gate, bounding rather than universalizing calibration. In post-trained Qwen/Qwen3-4B, frozen final-token h28 affine maps have mean held-entity error .519, versus .398 for within-test-domain cross-fit. Seven randomized entity splits and map geometry do not support a purely entity-specific account. Earlier h4/h16 layers fit one-step transitions better, but h4 conflict-state decoding is weak and lexical controls remain unresolved. Three matched intervention datasets regenerated from one frozen checkpoint show causal effects only at h28/h36. Outcome-aware refitting improves h28 one-step error to .474 (.469 with weighting), yet no refit passes composition. Learned finite worlds likewise preserve relative algebraic signals or shared charts without held-source affine closure. Within the tested carriers, state availability, causal use, local geometry, and reusable closure are separable. The result is limited to one pretrained model, sampled final-token layers, two finite worlds, and the tested affine or diagnostic function classes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 16 references that are in the map. First posted on arXiv 13 Aug 2026.
Builds on
15 of 16 references.
References · 15
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2023In-Context Learning Creates Task Vectorscited by 302
- 2023Function Vectors in Large Language Modelscited by 292
- 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
- 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
- 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
- 2024Evaluating the World Model Implicit in a Generative Modelcited by 107
- 2017Simulating Action Dynamics with Neural Process Networkscited by 53
- 2025Language Models use Lookbacks to Track Beliefscited by 36
- 2025(How) Do Language Models Track State?cited by 31
- 2025How Do Language Models Compose Functions?cited by 7
3 more
- 2025AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Modelscited by 7
- 2026Representational Homomorphism Predicts and Improves Compositional Generalization In Transformer Language Modelcited by 2
- 2026A Held-Out Transition-Pair Falsifier for Long-Horizon Non-Abelian State Trackingcited by 2
Built on by
Not cited yet.