mlsanityData to 14 Oct 2026

Beyond Forgetting: Diagnosing and Harnessing Shared Reasoning in Continual RLVR

arXiv 2608.18574 ↗First posted 19 Aug 2026cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) commonly post-trains reasoning models on multiple tasks, while rerunning multitask RLVR (MTRL) as new tasks are added makes capability expansion costly. We therefore study continual RLVR, which updates the existing model as each task arrives. The central question is whether a model updated this way can perform as well as a jointly trained model. To answer this question, we introduce Continual Reasoning Gym, a continual-RLVR environment that organizes text and visual reasoning tasks into five task sequences. In this setting, we identify two key observations: Sequential RLVR exhibits modest forgetting, yet its final performance remains below that of MTRL. To understand the latter, we decompose final performance and show that forgetting accounts for only part of the gap. To explain the former, we identify shared reasoning: transferable reasoning structure allows training on one task to support others on average. We therefore introduce Continual Prompt Replay (CPR), which harnesses shared reasoning to improve learning on the arriving and future tasks by replaying previous-task prompts and regenerating their responses with the current policy. On average, only CPR reaches MTRL-level performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 38 references that are in the map. First posted on arXiv 19 Aug 2026.

Builds on

21 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective30 Jun
  2. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective17 Jun
  3. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun

Other references · 18

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  5. 2025Muon is Scalable for LLM Trainingcited by 428
  6. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
  7. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  8. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
  9. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  10. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learningcited by 72
  11. 2022CLiMB: A Continual Learning Benchmark for Vision-and-Language Taskscited by 68
  12. 2023TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Modelscited by 66
6 more
  1. 2025Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learningcited by 15
  2. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 11
  3. 2024ViLCo-Bench: VIdeo Language COntinual learning Benchmarkcited by 9
  4. 2026FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoffcited by 7
  5. 2025Achieving Upper Bound Accuracy of Joint Training in Continual Learningcited by 4
  6. 2026SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning Portability: Guiding Continual Learning for MLLMs in the RLVR Era
  2. 2026RL Forgets! Towards Continual Policy Optimization
  3. 2025Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models
  4. 2026Detecting RLVR Training Data via Structural Convergence of Reasoning
  5. 2026Learning to Solve, Forgetting to Retain: Correct-Set Turnover in RLVR
  6. 2026Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning
  7. 2026Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning
  8. 2025Boosting Reasoning in Large Multimodal Models via Activation Replay