mlsanityData to 14 Oct 2026

Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models

arXiv 2510.21978 ↗First posted 24 Oct 2025cs.LG cs.AIcited by 5

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning and has become a standard post-training paradigm for contemporary language and vision-language models. However, the RLVR recipe introduces a significant risk of capability regression, in which models forget foundational skills after prolonged training without employing regularization strategies. We empirically confirm this concern, observing that open-source reasoning models suffer performance degradation on core capabilities such as perception and faithfulness. While imposing regularization terms like KL divergence can help prevent deviation from the base model, these terms are computed on the current task and therefore do not guarantee preservation of broader knowledge. Meanwhile, commonly used experience replay across heterogeneous domains makes it nontrivial to decide how much training emphasis each objective should receive. To address this, we propose RECAP-a replay strategy with dynamic objective reweighting for general knowledge preservation. Our reweighting mechanism adapts online using short-horizon signals of convergence and instability, shifting the post-training focus away from saturated objectives and toward underperforming or volatile ones. Our method is end-to-end and readily applicable to existing RLVR pipelines without training additional models or heavy tuning. Extensive experiments on benchmarks using Qwen2.5-VL-3B and Qwen2.5-VL-7B demonstrate the effectiveness of our method, which not only preserves general capabilities but also improves reasoning by enabling more flexible trade-offs among in-task rewards.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 101 of its 123 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

101 of 123 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Rethinking Entropy Interventions in RLVR: An Entropy Change Perspective11 Oct
  2. 2025RLEP: Reinforcement Learning with Experience Replay for LLM Reasoning10 Jul
  3. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning24 Jun
  4. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models30 May
  5. 2025Maximizing Confidence Alone Improves Reasoning28 May
  6. 2025Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models15 May
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  8. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  10. 2025Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models16 Mar
  11. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  12. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
4 more
  1. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning10 Oct
  2. 2023Let's Verify Step by Step31 May
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  4. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 85

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
38 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2016Learning without Forgettingcited by 2,496
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  11. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  12. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  13. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  14. 2017Mixed Precision Trainingcited by 1,156
  15. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  16. 2018Experience Replay for Continual Learningcited by 1,095
  17. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  18. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  19. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  20. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  21. 2016A Diagram Is Worth A Dozen Imagescited by 804
  22. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  23. 2017Deep Reinforcement Learning that Matterscited by 754
  24. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  25. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  26. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  27. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  28. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  29. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  30. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  31. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  32. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  33. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  34. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  35. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  36. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  37. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  38. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 323

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View
  2. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement
  3. 2025Boosting Reasoning in Large Multimodal Models via Activation Replay
  4. 2026Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioning
  5. 2026SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs
  6. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origin
  7. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  8. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models