mlsanityData to 14 Oct 2026

Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap

arXiv 2605.24432 ↗First posted 23 May 2026cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Large Language Model (LLM) interactions are typically underspecified, with users clarifying all necessary details across multiple conversational turns. Yet recent work shows that LLMs perform far worse in this multi-turn setting than in a single turn with same information being available at once, a phenomenon termed "Lost-in-Conversation." However, bridging this gap effectively remains an open problem. Here we introduce Found in Conversation (FiC), a training framework where a model teaches itself to find and recover its single-turn competence given underspecified multi-turn prompts. We develop View-Asymmetric Self-Distillation, which distills across two views of the same task information--single-turn view for the teacher, multi-turn view for the student--transferring strong single-turn behavior into weak multi-turn behavior. This requires no stronger external teacher, which is unavailable as even frontier LLMs exhibit this gap. Across model families (Llama, Qwen, Phi, and OLMo) and sizes (3B-14B), FiC recovers at least 92% of single-turn performance and reaches 100% on two Llama backbones, yielding more efficient and helpful multi-turn conversations with single-turn capabilities intact.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

56 of 66 references.

References · 56

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  9. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  10. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  11. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  12. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
38 more
  1. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  2. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  3. 2016Sequence-Level Knowledge Distillationcited by 801
  4. 2024Phi-4 Technical Reportcited by 711
  5. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  6. 2018Born Again Neural Networkscited by 546
  7. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  8. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
  9. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 455
  10. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  11. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  12. 20242 OLMo 2 Furiouscited by 340
  13. 2026Reinforcement Learning via Self-Distillationcited by 332
  14. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 290
  15. 2026Self-Distillation Enables Continual Learningcited by 247
  16. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  17. 2022Learning by Distilling Contextcited by 138
  18. 2023Training Language Models with Language Feedback at Scalecited by 128
  19. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 124
  20. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  21. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 104
  22. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 98
  23. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
  24. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  25. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 86
  26. 2024Multi-turn Reinforcement Learning from Preference Human Feedbackcited by 67
  27. 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 64
  28. 2024Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Trainingcited by 57
  29. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 53
  30. 2025Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Managementcited by 51
  31. 2025On the generalization of language models from in-context learning and finetuning: a controlled studycited by 40
  32. 2023ClarifyGPT: Empowering LLM-based Code Generation with Intention Clarificationcited by 38
  33. 2024Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHFcited by 30
  34. 2025A Survey on Multi-Turn Interaction Capabilities of Large Language Modelscited by 30
  35. 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 24
  36. 2024Efficient Knowledge Injection in LLMs via Self-Distillationcited by 24
  37. 2024On the Brittle Foundations of ReAct Prompting for Agentic Large Language Modelscited by 17
  38. 2026Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversationcited by 9

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Historiescited by 0
  2. 2026You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialoguecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models
  2. 2026Beyond Continuity: Challenges of Context Switching in Multi-Turn Dialogue with LLMs
  3. 2025Multi-Turn Puzzles: Evaluating Interactive Reasoning and Strategic Dialogue in LLMs
  4. 2026SOMA: Efficient Multi-turn LLM Serving via Small Language Model
  5. 2026Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
  6. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey
  7. 2025ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Models
  8. 2023Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models