mlsanityData to 7 Oct 2026

Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models

arXiv 2507.18014 ↗First posted 24 Jul 2025cs.LGcited by 6

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Fine-tuning large language models (LLMs) for reasoning tasks using reinforcement learning methods like Group Relative Policy Optimization (GRPO) is computationally expensive. To address this, we propose a predictive framework that models training dynamics and helps optimize resource usage. Through experiments on Llama and Qwen models (3B 8B), we derive an empirical scaling law based on model size, initial performance, and training progress. This law predicts reward trajectories and identifies three consistent training phases: slow start, rapid improvement, and plateau. We find that training beyond certain number of an epoch offers little gain, suggesting earlier stopping can significantly reduce compute without sacrificing performance. Our approach generalizes across model types, providing a practical guide for efficient GRPO-based fine-tuning.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 23 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

16 of 23 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  2. 2025Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning10 Feb
  3. 2025Process Reinforcement through Implicit Rewards3 Feb
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  5. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 11

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 287
  9. 2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
  10. 2025Tina: Tiny Reasoning Models via LoRAcited by 23
  11. 2025Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performancecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  2. 2025Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughtscited by 12
  3. 2026GanitLLM: Difficulty-Aware Bengali Mathematical Reasoning through Curriculum-GRPOcited by 6
  4. 2026MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweightingcited by 2
  5. 2025Butter-Bench: Evaluating LLM Controlled Robots for Practical Intelligencecited by 1
  6. 2026Predictable GRPO: A Closed-Form Model of Training Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GRPO Training Dynamics for Small Language Models
  2. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?
  3. 2025Comparative Analysis and Parametric Tuning of PPO, GRPO, and DAPO for LLM Reasoning Enhancement
  4. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  5. 2025Training Language Models to Reason Efficiently
  6. 2025Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning
  7. 2025On Predictability of Reinforcement Learning Dynamics for Large Language Models
  8. 2026GRPO and Reflection Reward for Mathematical Reasoning in Large Language Models