mlsanityData to 7 Oct 2026

How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study

arXiv 2504.00829 ↗First posted 1 Apr 2025cs.CLcited by 14

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Enhancing the reasoning capabilities of Large Language Models (LLMs) with efficiency and scalability remains a fundamental challenge in artificial intelligence research. This paper presents a rigorous experimental investigation into how difficulty-aware staged reinforcement learning (RL) strategies can substantially improve LLM reasoning performance. Through systematic analysis, we demonstrate that strategically selecting training data according to well-defined difficulty levels markedly enhances RL optimization. Moreover, we introduce a staged training methodology, progressively exposing models to increasingly challenging tasks, further amplifying reasoning capabilities. Our findings reveal significant cross-domain benefits when simultaneously training models on mathematical reasoning and code generation tasks. Notably, our proposed approach enables a 1.5B parameter model to achieve an accuracy of 42.3% on the AIME-2024 benchmark, 89.5% on the MATH-500 benchmark. These results underscore the efficacy of our method in advancing the reasoning proficiency of LLMs. We will open-source our datasets on GitHub and Hugging Face.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 22 references that are in the map. First posted on arXiv 1 Apr 2025.

Builds on

13 of 22 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  2. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  3. 2025Process Reinforcement through Implicit Rewards3 Feb

Other references · 10

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2025LIMO: Less is More for Reasoningcited by 488
  9. 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Trainingcited by 49
  10. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representationscited by 26
  2. 2025Difficulty-Aware Agentic Orchestration for Query-Specific Multi-Agent Workflowscited by 23
  3. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Trainingcited by 17
  4. 2025Reinforcement Learning for Out-of-Distribution Reasoning in LLMs: An Empirical Study on Diagnosis-Related Group Codingcited by 7
  5. 2025DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimizationcited by 7
  6. 2025More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoningcited by 4
  7. 2025Verifying Meta-Awareness via Predictive Rewards in Reasoning Modelscited by 3
  8. 2025Shorter but not Worse: Frugal Reasoning via Easy Samples as Length Regularizers in Math RLVRcited by 2
  9. 2026TACLer: Tailored Curriculum Reinforcement Learning for Efficient Reasoningcited by 2
  10. 2026LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agentscited by 2
  11. 2025FEVO: Financial Knowledge Expansion and Reasoning Evolution for Large Language Modelscited by 1
  12. 2026Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signalscited by 0
2 more
  1. 2026Adaptive Loss Balancing for Noise-Robust GRPO in Generative Recommendationcited by 0
  2. 2026Dynamic Important Example Mining for Reinforcement Finetuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  2. 2025Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  4. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  5. 2025Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation
  6. 2025LogicPuzzleRL: Cultivating Robust Mathematical Reasoning in LLMs via Reinforcement Learning
  7. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
  8. 2025Rethinking the Sampling Criteria in Reinforcement Learning for LLM Reasoning: A Competence-Difficulty Alignment Perspective