mlsanityData to 14 Oct 2026

Multi-Turn Reasoning When Context Arrives in Pieces: Scalable Sharding and Memory-Augmented RL

arXiv 2606.12941 ↗First posted 11 Jun 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

When a user reveals task-critical information across several conversation turns, LLM accuracy drops by up to 65% despite full context availability. We show that this Lost in Conversation degradation can be substantially mitigated by training models to maintain a compact rolling memory instead of attending to a growing history. To make such training scalable, we introduce a low-cost sharding pipeline that converts single-turn QA datasets into multi-turn fragmented-information episodes, eliminating the need for hours of manual annotation. Training only on sharded GSM8K, our memory-augmented policy significantly improves multi-turn accuracy and generalises zero-shot to harder math and out-of-domain long-context QA. Moreover, memory-trained models outperform full-history baselines even when given the full history at test time, suggesting that learning to compress induces more robust incremental reasoning than full-context exposure alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 24 of its 28 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

24 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent3 Jul

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  11. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
11 more
  1. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  5. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  7. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  8. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  9. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
  10. 2024MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Modelscited by 81
  11. 2025Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewardscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025CogMem: A Cognitive Memory Architecture for Sustained Multi-Turn Reasoning in Large Language Models
  2. 2026Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
  3. 2026Beyond Continuity: Challenges of Context Switching in Multi-Turn Dialogue with LLMs
  4. 2025One-Pass to Reason: Token Duplication and Block-Sparse Mask for Efficient Fine-Tuning on Multi-Turn Reasoning
  5. 2026PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory
  6. 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues
  7. 2026CMT-RAG: Complementary Memory Traces for Multi-turn Multi-hop RAG
  8. 2026Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap