mlsanityData to 7 Oct 2026

What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction

arXiv 2609.05882 ↗First posted 5 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-turn interaction creates a feedback process in which an LLM's previous responses become context for later behavior. Prior work shows substantial multi-turn degradation and that assistant-generated history can affect later behavior. However, it remains unclear how these effects manifest across models, tasks, turns, and inside a model. We study these gaps across six task families and five models. Degradation from fully specified single-turn input (FULL) to progressively revealed multi-turn interaction (SHARDED) is clearly task- and model-dependent, and stronger one-shot performance does not imply greater interaction robustness. We then retrospectively analyze completed SHARDED conversations by replaying the user messages already observed in each trajectory while editing only assistant-generated history. Replacing prior assistant responses with neutral content (termed neutralization) changes downstream min-max normalized performance by +.027 across 2,973 trajectories. On a prespecified length-controlled subset, short and length-matched neutralization yield nearly identical effects (+.069 versus +.068), showing that simple context shortening is insufficient to explain the effect of history editing. Turn Surgery further intervenes on one assistant turn at a time. Among 237 selected degraded trajectories, 63.7% contain at least one beneficial intervention, while most tested positions remain unchanged; for binary tasks, 48.4% admit a fail-to-success reversal. An open-weight case study links behaviorally consequential history changes to measurable downstream state differences, but finds task-dependent rather than universal internal signatures. Overall, assistant-generated history has active but selective effects on multi-turn performance, motivating selective rather than uniform history management.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 5 Sep 2026.

Builds on

27 of 39 references.

References · 27

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  8. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  9. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  10. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  11. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  12. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
15 more
  1. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 247
  2. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 171
  3. 2025Where LLM Agents Fail and How They can Learn From Failurescited by 118
  4. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  5. 2024On the Multi-turn Instruction Following for Conversational Web Agentscited by 42
  6. 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systemscited by 23
  7. 2026Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failurescited by 18
  8. 2026CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failurescited by 17
  9. 2026Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversationcited by 7
  10. 2026Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interactioncited by 7
  11. 2025ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Modelscited by 6
  12. 2025Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewardscited by 6
  13. 2026Mitigating Conversational Inertia in Multi-Turn Agentscited by 5
  14. 2026MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillationcited by 3
  15. 2026MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversationcited by 2

Built on by

Not cited yet.