Learning to Solve, Forgetting to Retain: Correct-Set Turnover in RLVR
Not on the map. Why?
Abstract (arXiv)
Reinforcement learning with verifiable rewards (RLVR) improves the ability of large language model, yet headline accuracy gains often conceal a hidden cost: previously solved problems quietly become unsolvable as training proceeds. We frame this phenomenon as correct-set turnover, representing the coupled dynamics of solution acquisition and regression over the mastered set. Under this view, retention becomes an explicit optimization target alongside acquisition. We analytically and empirically establish the repair-window principle: the cost of restoring a regressed prompt grows sharply with review delay, defining a low-cost window that standard RLVR pipelines fail to exploit. To address this, we propose textbfmethod, a retention-aware review mechanism that tracks mastered prompts and periodically reintroduces them to remind the model of previous solutions. By utilizing pre-rollout batch replacement, method incurs zero additional rollout overhead. Evaluated across 20 benchmarks spanning image-text, video, and text-only tasks with Qwen3-VL and Qwen2.5-Math, method consistently improves performance over GRPO, DAPO, and replay baselines, demonstrating robust generalizability across modalities and algorithms.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 2 Jun 2026.
Builds on
43 of 55 references.
References · 43
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2015Prioritized Experience Replaycited by 1,070
31 more
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
- 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
- 2025Dynamic Early Exit in Reasoning Modelscited by 217
- 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
- 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
- 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
- 2025MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Modelscited by 69
- 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Modelscited by 56
- 2025ExGRPO: Learning to Reason from Experiencecited by 49
- 2025No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shapingcited by 48
- 2025RePO: Replay-Enhanced Policy Optimizationcited by 36
- 2025RLEP: Reinforcement Learning with Experience Replay for LLM Reasoningcited by 35
- 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 35
- 2025Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reusecited by 28
- 2026MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methodscited by 27
- 2025VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal Understanding in Videoscited by 23
- 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
- 2025Emergent Slow Thinking in LLMs as Inverse Tree Freezingcited by 8
- 2025Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Modelscited by 5
- 2026System 1&2 Synergy via Dynamic Model Interpolationcited by 3
- 2026Probing RLVR training instability through the lens of objective-level hackingcited by 3
- 2026Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learningcited by 3
- 2026EasyVideoR1: Easier RL for Video Understandingcited by 1
- 2026MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026When to Stop Reusing: Dynamic Gradient Gating for Sample-Efficient RLVR
- 2025On the optimization dynamics of RLVR: Gradient gap and step size thresholds
- 2025The Path Not Taken: RLVR Provably Learns Off the Principals
- 2026Linear Dynamics in the RLVR Training of Large Language Models
- 2026Beyond Forgetting: Diagnosing and Harnessing Shared Reasoning in Continual RLVR
- 2026What are Key Factors for Updates in RL for LLM Reasoning?
- 2026Spend Your Rollouts Where It Counts: Rollout Allocation for Group-Based RL Post-Training
- 2026When RLVR Shrinks the Reasoning Boundary: Diagnosing Pass@k Inversion