mlsanityData to 7 Oct 2026

Learning to Solve, Forgetting to Retain: Correct-Set Turnover in RLVR

arXiv 2606.03087 ↗First posted 2 Jun 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) improves the ability of large language model, yet headline accuracy gains often conceal a hidden cost: previously solved problems quietly become unsolvable as training proceeds. We frame this phenomenon as correct-set turnover, representing the coupled dynamics of solution acquisition and regression over the mastered set. Under this view, retention becomes an explicit optimization target alongside acquisition. We analytically and empirically establish the repair-window principle: the cost of restoring a regressed prompt grows sharply with review delay, defining a low-cost window that standard RLVR pipelines fail to exploit. To address this, we propose textbfmethod, a retention-aware review mechanism that tracks mastered prompts and periodically reintroduces them to remind the model of previous solutions. By utilizing pre-rollout batch replacement, method incurs zero additional rollout overhead. Evaluated across 20 benchmarks spanning image-text, video, and text-only tasks with Qwen3-VL and Qwen2.5-Math, method consistently improves performance over GRPO, DAPO, and replay baselines, demonstrating robust generalizability across modalities and algorithms.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

43 of 55 references.

References · 43

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  10. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  11. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  12. 2015Prioritized Experience Replaycited by 1,070
31 more
  1. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  2. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  3. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  4. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  5. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  6. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  7. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  8. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  9. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  10. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  11. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
  12. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
  13. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  14. 2025MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Modelscited by 69
  15. 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Modelscited by 56
  16. 2025ExGRPO: Learning to Reason from Experiencecited by 49
  17. 2025No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shapingcited by 48
  18. 2025RePO: Replay-Enhanced Policy Optimizationcited by 36
  19. 2025RLEP: Reinforcement Learning with Experience Replay for LLM Reasoningcited by 35
  20. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 35
  21. 2025Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reusecited by 28
  22. 2026MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methodscited by 27
  23. 2025VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal Understanding in Videoscited by 23
  24. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  25. 2025Emergent Slow Thinking in LLMs as Inverse Tree Freezingcited by 8
  26. 2025Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Modelscited by 5
  27. 2026System 1&2 Synergy via Dynamic Model Interpolationcited by 3
  28. 2026Probing RLVR training instability through the lens of objective-level hackingcited by 3
  29. 2026Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learningcited by 3
  30. 2026EasyVideoR1: Easier RL for Video Understandingcited by 1
  31. 2026MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026When to Stop Reusing: Dynamic Gradient Gating for Sample-Efficient RLVR
  2. 2025On the optimization dynamics of RLVR: Gradient gap and step size thresholds
  3. 2025The Path Not Taken: RLVR Provably Learns Off the Principals
  4. 2026Linear Dynamics in the RLVR Training of Large Language Models
  5. 2026Beyond Forgetting: Diagnosing and Harnessing Shared Reasoning in Continual RLVR
  6. 2026What are Key Factors for Updates in RL for LLM Reasoning?
  7. 2026Spend Your Rollouts Where It Counts: Rollout Allocation for Group-Based RL Post-Training
  8. 2026When RLVR Shrinks the Reasoning Boundary: Diagnosing Pass@k Inversion