mlsanityData to 7 Oct 2026

Document Reconstruction Unlocks Scalable Long-Context RLVR

arXiv 2602.08237 ↗First posted 9 Feb 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards~(RLVR) has become a prominent paradigm to enhance the capabilities (i.e. long-context) of Large Language Models~(LLMs). However, it often relies on gold-standard answers or explicit evaluation rubrics provided by powerful teacher models or human experts, which are costly and time-consuming. In this work, we investigate unsupervised approaches to enhance the long-context capabilities of LLMs, eliminating the need for heavy human annotations or teacher models' supervision. Specifically, we first replace a few paragraphs with special placeholders in a long document. LLMs are trained through reinforcement learning to reconstruct the document by correctly identifying and sequencing missing paragraphs from a set of candidate options. This training paradigm enables the model to capture global narrative coherence, significantly boosting long-context performance. We validate the effectiveness of our method on two widely used benchmarks, RULER and LongBench~v2. While acquiring noticeable gains on RULER, it can also achieve a reasonable improvement on LongBench~v2 without any manually curated long-context QA data. Furthermore, we conduct extensive ablation studies to analyze the impact of reward design, data curation strategies, training schemes, and data scaling effects on model performance. We publicly release our code, data, and models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

42 of 51 references.

References · 42

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  8. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  9. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  10. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  11. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  12. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
30 more
  1. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  2. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  3. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
  4. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  5. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
  6. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 187
  7. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 142
  8. 2024How to Train Long-Context Language Models (Effectively)cited by 141
  9. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 118
  10. 2023PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Trainingcited by 109
  11. 2025Reinforcement Learning with Rubric Anchorscited by 108
  12. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
  13. 2025Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipelinecited by 79
  14. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 72
  15. 2024Training-Free Long-Context Scaling of Large Language Modelscited by 61
  16. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Modelscited by 56
  17. 2025QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learningcited by 46
  18. 2026Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Trainingcited by 45
  19. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  20. 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
  21. 2024LongReward: Improving Long-context Large Language Models with AI Feedbackcited by 34
  22. 2024Making Long-Context Language Models Better Multi-Hop Reasonerscited by 25
  23. 2025LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contextscited by 25
  24. 2025LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimizationcited by 24
  25. 2025Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analyticscited by 20
  26. 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL Systemcited by 19
  27. 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learningcited by 13
  28. 2026VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agentscited by 9
  29. 2026Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Textcited by 9
  30. 2025Do Students Debias Like Teachers? On the Distillability of Bias Mitigation Methodscited by 2

Built on by

Not cited yet.