mlsanityData to 14 Oct 2026

Limits of Generalization in RLVR: Two Case Studies in Mathematical Reasoning

arXiv 2510.27044 ↗First posted 30 Oct 2025cs.LGcited by 6

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Mathematical reasoning is a central challenge for large language models (LLMs), requiring not only correct answers but also faithful reasoning processes. Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising approach for enhancing such capabilities; however, its ability to foster genuine reasoning remains unclear. We investigate RLVR on two combinatorial problems with fully verifiable solutions: Activity Scheduling and the Longest Increasing Subsequence, using carefully curated datasets with unique optima. Across multiple reward designs, we find that RLVR improves evaluation metrics but often by reinforcing superficial heuristics rather than acquiring new reasoning strategies. These findings highlight the limits of RLVR generalization, emphasizing the importance of benchmarks that disentangle genuine mathematical reasoning from shortcut exploitation and provide faithful measures of progress. Code available at https://github.com/xashru/rlvr-seq-generalization.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 23 references that are in the map. First posted on arXiv 30 Oct 2025.

Builds on

17 of 23 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun
  2. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  3. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning2 Jun
  4. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  5. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan

Other references · 9

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2024OpenAI o1 System Cardcited by 1,244
  7. 2025Open Problems in Mechanistic Interpretabilitycited by 179
  8. 2025Competitive Programming with Large Reasoning Modelscited by 119
  9. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origincited by 97

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMscited by 14
  2. 2026CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVRcited by 5
  3. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  4. 2026GeoRA: Geometry-Aware Low-Rank Adaptation for RLVRcited by 4
  5. 2026From Reasoning Chains to Verifiable Subproblems: Curriculum Reinforcement Learning Enables Credit Assignment for LLM Reasoningcited by 2
  6. 2026QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewardscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Generalization of RLVR Using Causal Reasoning as a Testbed
  2. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
  3. 2026Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards
  4. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  5. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  6. 2026SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions
  7. 2026Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes
  8. 2025Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards