# Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models

arXiv: https://arxiv.org/abs/2507.18014 · HTML: https://mlsanity.com/paper/2507.18014/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://mlsanity.com/paper/2503.18892/) (2025)
- [Process Reinforcement through Implicit Rewards](https://mlsanity.com/paper/2502.01456/) (2025)
- [OpenThoughts: Data Recipes for Reasoning Models](https://mlsanity.com/paper/2506.04178/) (2025)
- [Enhancing LLM Reasoning with Reward-guided Tree Search](https://mlsanity.com/paper/2411.11694/) (2024)
- [Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning](https://mlsanity.com/paper/2502.06781/) (2025)
- [Tina: Tiny Reasoning Models via LoRA](https://mlsanity.com/paper/2504.15777/) (2025)
- [Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performance](https://mlsanity.com/paper/2506.05748/) (2025)

## Built on by (most cited first, 6 of 6)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughts](https://mlsanity.com/paper/2509.21743/) (2025)
- [GanitLLM: Difficulty-Aware Bengali Mathematical Reasoning through Curriculum-GRPO](https://mlsanity.com/paper/2601.06767/) (2026)
- [Butter-Bench: Evaluating LLM Controlled Robots for Practical Intelligence](https://mlsanity.com/paper/2510.21860/) (2025)
- [MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting](https://mlsanity.com/paper/2601.09085/) (2026)
- [Predictable GRPO: A Closed-Form Model of Training Dynamics](https://mlsanity.com/paper/2606.30789/) (2026)

## Abstract (arXiv)

Fine-tuning large language models (LLMs) for reasoning tasks using reinforcement learning methods like Group Relative Policy Optimization (GRPO) is computationally expensive. To address this, we propose a predictive framework that models training dynamics and helps optimize resource usage. Through experiments on Llama and Qwen models (3B 8B), we derive an empirical scaling law based on model size, initial performance, and training progress. This law predicts reward trajectories and identifies three consistent training phases: slow start, rapid improvement, and plateau. We find that training beyond certain number of an epoch offers little gain, suggesting earlier stopping can significantly reduce compute without sacrificing performance. Our approach generalizes across model types, providing a practical guide for efficient GRPO-based fine-tuning.