# MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation

arXiv: https://arxiv.org/abs/2605.27186 · HTML: https://mlsanity.com/paper/2605.27186/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 31 of its 62 references that are in the map.
- First posted on arXiv 26 May 2026.

## Placed in

- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 30 of 31)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [On-Policy Context Distillation for Language Models](https://mlsanity.com/paper/2602.12275/) (2026)
- [Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation](https://mlsanity.com/paper/2605.18740/) (2026)
- [TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents](https://mlsanity.com/paper/2604.24005/) (2026)
- [The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes](https://mlsanity.com/paper/2605.11182/) (2026)
- [SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting](https://mlsanity.com/paper/2604.10688/) (2026)
- [OPSDL: On-Policy Self-Distillation for Long-Context Language Models](https://mlsanity.com/paper/2604.17535/) (2026)
- [AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thought](https://mlsanity.com/paper/2501.16154/) (2025)
- [CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation](https://mlsanity.com/paper/2507.14239/) (2025)
- [Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation](https://mlsanity.com/paper/2602.07338/) (2026)
- [Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interaction](https://mlsanity.com/paper/2603.04783/) (2026)
- [Mitigating Conversational Inertia in Multi-Turn Agents](https://mlsanity.com/paper/2602.03664/) (2026)
- [Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards](https://mlsanity.com/paper/2510.18731/) (2025)
- [SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Following](https://mlsanity.com/paper/2605.06353/) (2026)
- [Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations](https://mlsanity.com/paper/2604.27093/) (2026)
- [Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation](https://mlsanity.com/paper/2604.28031/) (2026)
- [When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction](https://mlsanity.com/paper/2605.12922/) (2026)
- [Grounded Continuation: A Linear-Time Runtime Verifier for LLM Conversations](https://mlsanity.com/paper/2605.14175/) (2026)
- [ICPO: Illocution-Calibrated Policy Optimization for Multi-Turn Conversation](https://mlsanity.com/paper/2601.15330/) (2026)
- [Pause or Fabricate? Training Language Models for Grounded Reasoning](https://mlsanity.com/paper/2604.19656/) (2026)

## Built on by (most cited first, 6 of 6)

- [Trajectory-Relative Hindsight Distillation for Agentic Reinforcement Learning](https://mlsanity.com/paper/2608.07371/) (2026)
- [What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction](https://mlsanity.com/paper/2609.05882/) (2026)
- [SMOPD: Selective Token-Entropy Masking for Dirty-History Multi-Turn On-Policy Self-Distillation](https://mlsanity.com/paper/2608.14647/) (2026)
- [You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialogue](https://mlsanity.com/paper/2610.06496/) (2026)
- [Privileged Context as Drift in On-Policy Self-Distillation](https://mlsanity.com/paper/2610.07842/) (2026)
- [State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents](https://mlsanity.com/paper/2610.13238/) (2026)

## Abstract (arXiv)

Large language models often solve tasks from a fully specified prompt but degrade when the same requirements unfold over multiple turns, known as the lost-in-conversation (LiC) gap. We trace part of this degradation to self-contamination: intermediate assistant replies enter later context and carry early deviations forward. Motivated by this mechanism, we propose MAIGO, an on-policy self-distillation method that reduces this contamination using history-cleaned references from the model's own policy. For middle turns, MAIGO removes prior assistant replies while preserving the user-visible sharded prefix; for answer turns, it distills from paired full-view references conditioned on the completed user-side dialogue. A reliability weight downweights middle-turn samples that disagree with the clean reference. MAIGO requires no verifier rewards, state labels, or inference-time scaffolding. Under the LiC paired-view protocol with deterministic verifiers, MAIGO improves Qwen2.5-7B-Instruct SHARDED accuracy from 52.8 to 66.1 and the SHARDED/FULL ratio from 66.5% to 84.1%, while keeping FULL accuracy within 2.3 points. These results show that self-contamination is a trainable component of the LiC gap.