mlsanityData to 14 Oct 2026

Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning

arXiv 2606.18831 ↗First posted 17 Jun 2026cs.CL cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Long-context reasoning is an essential capability for large language models, particularly when they are deployed as autonomous agents that must reason over lengthy trajectories. Reinforcement learning (RL) has recently emerged as a dominant paradigm for improving this ability, yet existing work largely focuses on reward engineering while diverse training data remains scarce. We revisit this problem from a data-centric perspective and show that a simple yet effective data recipe alone, paired with a minimal outcome-based GRPO setup, suffices to substantially improve long-context reasoning. Our recipe targets three complementary task families -- retrieval, multi-evidence synthesis, and reasoning -- for which we construct and curate eight datasets totaling ~14K examples. Experiments on three models (Qwen3-4B/8B/30B-A3B) yield average gains of +7.2/+3.2/+6.4 points across seven long-context benchmarks, surpassing prior RL training sets. We further demonstrate that these gains transfer to agentic tasks, where continuing RL training on an agent-tuned model with our data recipe improves GAIA by +4.8 and BrowseComp by +7.0 points. We will release our datasets to facilitate future research.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 39 of its 53 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

39 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning16 Apr
  2. 2026LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards2 Mar

Other references · 37

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  6. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  7. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  8. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  11. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  12. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
25 more
  1. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  2. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  3. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  4. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 257
  5. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 162
  6. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 127
  7. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 126
  8. 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agentscited by 92
  9. 2024Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queriescited by 88
  10. 2024DocFinQA: A Long-Context Financial Reasoning Datasetcited by 60
  11. 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RLcited by 55
  12. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansioncited by 50
  13. 2025QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learningcited by 50
  14. 2024NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densitiescited by 32
  15. 2024Large Language Models Can Self-Improve in Long-context Reasoningcited by 29
  16. 2025LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contextscited by 29
  17. 2025QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Managementcited by 27
  18. 2026Benchmark Test-Time Scaling of General LLM Agentscited by 26
  19. 2024Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystackcited by 19
  20. 2026AgentCPM-Explore: Realizing Long-Horizon Deep Exploration for Edge-Scale Agentscited by 12
  21. 2026LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewardscited by 10
  22. 2025NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contextscited by 8
  23. 2026Evidence-Augmented Policy Optimization with Reward Co-Evolution for Long-Context Reasoningcited by 7
  24. 2026A Decomposition Perspective to Long-context Reasoning for LLMscited by 1
  25. 2026RecaLLM: Addressing the Lost-in-Thought Phenomenon with Explicit In-Context Retrievalcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  2. 2026Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
  3. 2026GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
  4. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
  5. 2024Improving Sample Efficiency of Reinforcement Learning with Background Knowledge from Large Language Models
  6. 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models
  7. 2025Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Models
  8. 2026LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards