mlsanityData to 14 Oct 2026

Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards

arXiv 2510.18731 ↗First posted 21 Oct 2025cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Large Language Models demonstrate strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC), a degradation in performance as information is revealed progressively in multi-turn settings. Motivated by the current progress on Reinforcement Learning with Verifiable Rewards (RLVR), we propose Curriculum Reinforcement Learning with Verifiable Accuracy and Abstention Rewards (RLAAR), a framework that encourages models not only to generate correct answers, but also to judge the solvability of questions in the multi-turn conversation setting. Our approach employs a competence-gated curriculum that incrementally increases dialogue difficulty (in terms of instruction shards), stabilizing training while promoting reliability. Using multi-turn, on-policy rollouts and a mixed-reward system, RLAAR teaches models to balance problem-solving with informed abstention, reducing premature answering behaviors that cause LiC. Evaluated on LiC benchmarks, RLAAR significantly mitigates LiC performance decay (62.6% to 75.1%) and improves calibrated abstention rates (33.5% to 73.4%). Together, these results provide a practical recipe for building multi-turn reliable and trustworthy LLMs.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

38 of 56 references.

References · 38

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  9. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  10. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  11. 2021Measuring Coding Challenge Competence With APPScited by 902
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
26 more
  1. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  2. 2018CoQA: A Conversational Question Answering Challengecited by 683
  3. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  4. 2018QuAC : Question Answering in Contextcited by 381
  5. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  6. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  7. 2023Do Large Language Models Know What They Don't Know?cited by 243
  8. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  9. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  10. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 156
  11. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  12. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  13. 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 77
  14. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 63
  15. 2024Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Modelscited by 57
  16. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 55
  17. 2024On the Multi-turn Instruction Following for Conversational Web Agentscited by 43
  18. 2025ChatBench: From Static Benchmarks to Human-AI Evaluationcited by 34
  19. 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
  20. 2024Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanationscited by 30
  21. 2024MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactionscited by 26
  22. 2024On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbotscited by 24
  23. 2024IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answeringcited by 19
  24. 2024LLMs and Memorization: On Quality and Specificity of Copyright Compliancecited by 16
  25. 2021What's The Latest? A Question-driven News Chatbotcited by 14
  26. 2025Synthetic Clarification and Correction Dialogues about Data-Centric Tasks -- A Teacher-Student Approachcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Adaptive Information Control for Search-Augmented LLM Reasoningcited by 11
  2. 2026Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversationcited by 9
  3. 2026MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillationcited by 6
  4. 2026Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gapcited by 2
  5. 2026What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interactioncited by 1
  6. 2026Multi-Turn Reasoning When Context Arrives in Pieces: Scalable Sharding and Memory-Augmented RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ICPO: Illocution-Calibrated Policy Optimization for Multi-Turn Conversation
  2. 2025SpeakRL: Synergizing Reasoning, Speaking, and Acting in Language Models with Reinforcement Learning
  3. 2026Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model
  4. 2026SeDT: Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability
  5. 2024Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF
  6. 2026Re2: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
  7. 2025AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting
  8. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learning