When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction
Not on the map. Why?
Abstract (arXiv)
Large language models can follow complex instructions in a single turn, yet over long multi-turn interactions they often lose the thread of instructions, persona, and rules. This degradation has been measured behaviorally but not mechanistically explained. We propose a channel-transition account: goal-defining tokens become less accessible through attention, while goal-related information may persist in residual representations. We introduce the Goal Accessibility Ratio (GAR), measuring attention from generated tokens to task-defining goal tokens, and combine it with sliding-window ablations and residual-stream probes. When attention to instructions closes, what survives reveals architecture. Across architectures, the transition yields qualitatively distinct failure modes: some models preserve goal-conditioned behavior at vanishing attention, others fail despite decodable residual goal information, and the layer at which this encoding emerges varies from 2 to 27. A within-model causal ablation that force-closes the attention channel in Mistral collapses recall from near-perfect to 11% on a 20-fact retention task and raises persona-constraint violations above an adversarial-pressure baseline without user pressure, with both effects emerging at the predictable crossover turn. Linear probes recover per-episode recall outcomes from residual representations with AUC up to 0.99 across all four primary architectures, while input embeddings remain at chance. Across architectures and model scales, the gap between attention loss and residual decodability predicts whether goal-conditioned behavior survives channel closure. We contribute GAR as a diagnostic, the channel-transition framework as a controlled mechanistic account, and a parametric prediction of failure timing under windowed attention closure.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 30 of its 36 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
30 of 36 references.
References · 30
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2023Mistral 7Bcited by 3,065
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2024Mixtral of Expertscited by 1,785
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
18 more
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
- 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
- 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
- 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 247
- 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 200
- 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
- 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 94
- 2024MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Modelscited by 78
- 2025Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coincited by 47
- 2025One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Frameworkcited by 9
- 2025Dynamic Context Adaptation for Consistent Role-Playing Agents with Retrieval-Augmented Generationscited by 4
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillationcited by 3
- 2026Measuring What Persists: Conditioning Mechanisms and a Geometric Framework for AI Agent Identitycited by 0
- 2026Geometry of Divergence: Tracking Hidden-State Trajectories for Adaptive Multi-Turn Reasoningcited by 0
- 2026Geometry of Divergence: Tracking Hidden-State Trajectories for Adaptive Multi-Turn Reasoning31 Aug 2026
- 2026Measuring What Persists: Conditioning Mechanisms and a Geometric Framework for AI Agent Identity20 Jun 2026
- 2026MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation26 May 2026