How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing
Not on the map. Why?
Abstract (arXiv)
When a decoder-only transformer is forced to process matched correct and incorrect single-token continuations of a factual query, the two pathways through hidden-state space diverge: displacement vectors from the query-only representation keep near-equal magnitude but rotate apart, with angular separation growing through mid-depth before late layers resolve an asymmetric outcome. A logit-lens preference in the incorrect run falls far below the equal-probability prior (roughly 11.5x more mass on the incorrect token than the correct one). We read this pattern, rotational divergence then late-layer asymmetric commitment, as the geometric signature of the model externally appearing to reject a wrong continuation, while staying explicit that it is observational, not causal: the incorrect run could equally reflect the model conforming to the token it is forced to carry, which only a random-token control can settle. It holds across six decoder-only transformers spanning four architecture families (Llama, Mistral, Gemma, StableLM) from 1B to 13B parameters; a seventh (Qwen2 1.5B) is flat under our protocol, plausibly a tokenizer artefact, leaving an emergence threshold open. Linear probes recover the distinction at intermediate depth, and cross-domain transfer is structurally asymmetric, a financial-medical corridor transferring far better than transport pairs. Where single-layer activation patching is cleanly interpretable (LLaMA-2 13B, Mistral 7B) it yields no layer band of consistent recovery; a third model (StableLM-2 1.6B) recovers uniformly above the ceiling, which we diagnose as a code-path artefact and exclude. Under this scoped null the late-layer asymmetry is not localized to a single component, fitting a distributed-by-trajectory account rather than single-layer localized recall. We document this with forced-completion probing across seven models, three domains, and 300 queries.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 25 Feb 2026.
Builds on
33 of 46 references.
References · 33
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Mistral 7Bcited by 3,065
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2024Qwen2 Technical Reportcited by 2,076
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
21 more
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2023The Internal State of an LLM Knows When It's Lyingcited by 644
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2021Knowledge Neurons in Pretrained Transformerscited by 590
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
- 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
- 2023Language Models Represent Space and Timecited by 306
- 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
- 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
- 2022Analyzing Transformers in Embedding Spacecited by 131
- 2022Evaluating the Factual Consistency of Large Language Models Through News Summarizationcited by 90
- 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
- 2024Stable LM 2 1.6B Technical Reportcited by 86
- 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.