Hallucination as Trajectory Commitment: Causal Evidence for Asymmetric Attractor Dynamics in Transformer Generation
Not on the map. Why?
Abstract (arXiv)
We present causal evidence that hallucination in autoregressive language models is an early trajectory commitment governed by asymmetric attractor dynamics. Using same-prompt bifurcation, in which we repeatedly sample identical inputs to observe spontaneous divergence, we isolate trajectory dynamics from prompt-level confounds. On Qwen2.5-1.5B across 61 prompts spanning six categories, 27 prompts (44.3%) bifurcate with factual and hallucinated trajectories diverging at the first generated token (KL = 0 at step 0, KL > 1.0 at step 1). Activation patching across 28 layers reveals a pronounced causal asymmetry: injecting a hallucinated activation into a correct trajectory corrupts output in 87.5% of trials (layer 20), while the reverse recovers only 33.3% (layer 24); both exceed the 10.4% baseline (p = 0.025) and 12.5% random-patch control. Window patching shows correction requires sustained multi-step intervention, whereas corruption needs only a single perturbation. Probing the prompt encoding itself, step-0 residual states predict per-prompt hallucination rate at Pearson r = 0.776 at layer 15 (p < 0.001 against a 1000-permutation null); unsupervised clustering identifies five regime-like groups (eta^2 = 0.55) whose saddle-adjacent cluster concentrates 12 of the 13 bifurcating false-premise prompts, indicating that the basin structure is organized around regime commitments fixed at prompt encoding. These findings characterize hallucination as a locally stable attractor basin: entry is probabilistic and rapid, exit demands coordinated intervention across layers and steps, and the relevant basins are selected by clusterable regimes already discernible at step 0.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 10 of its 19 references that are in the map. First posted on arXiv 16 Apr 2026.
Builds on
10 of 19 references.
References · 10
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
- 2023The Internal State of an LLM Knows When It's Lyingcited by 659
- 2024How to use and interpret activation patchingcited by 214
- 2026Projected Autoregression: Autoregressive Language Generation in Continuous State Spacecited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers
- 2026Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models
- 2026TRACESTEER: Trace Driven Activation Steering for Hallucination Mitigation in Diffusion Language Models
- 2026Action Hallucination in Generative Vision-Language-Action Models
- 2025Transformer Dynamics: A neuroscientific approach to interpretability of large language models
- 2025HEAL: An Empirical Study on Hallucinations in Embodied Agents Driven by Large Language Models
- 2023Do Androids Know They're Only Dreaming of Electric Sheep?
- 2026Lyapunov Probes for Hallucination Detection in Large Foundation Models