# Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning

arXiv: https://arxiv.org/abs/2509.25300 · HTML: https://mlsanity.com/paper/2509.25300/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 39 of its 57 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 30 of 39)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://mlsanity.com/paper/2503.18892/) (2025)
- [MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning](https://mlsanity.com/paper/2309.05653/) (2023)
- [The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models](https://mlsanity.com/paper/2505.22617/) (2025)
- [Scaling Relationship on Learning Mathematical Reasoning with Large Language Models](https://mlsanity.com/paper/2308.01825/) (2023)
- [Reinforcement Learning for Reasoning in Large Language Models with One Training Example](https://mlsanity.com/paper/2504.20571/) (2025)
- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models](https://mlsanity.com/paper/2505.24864/) (2025)
- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [Scaling Laws and Interpretability of Learning from Repeated Data](https://mlsanity.com/paper/2205.10487/) (2022)
- [From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review](https://mlsanity.com/paper/2504.19678/) (2025)
- [LIMR: Less is More for RL Scaling](https://mlsanity.com/paper/2502.11886/) (2025)
- [Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline](https://mlsanity.com/paper/2507.15855/) (2025)
- [SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines](https://mlsanity.com/paper/2502.14739/) (2025)
- [Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective](https://mlsanity.com/paper/2506.14965/) (2025)
- [Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving](https://mlsanity.com/paper/2505.07773/) (2025)
- [Scaling Laws for Downstream Task Performance of Large Language Models](https://mlsanity.com/paper/2402.04177/) (2024)

## Built on by (most cited first, 29 of 29)

- [Your Group-Relative Advantage Is Biased](https://mlsanity.com/paper/2601.08521/) (2026)
- [Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation](https://mlsanity.com/paper/2605.11739/) (2026)
- [Understanding LLM Reasoning for Abstractive Summarization](https://mlsanity.com/paper/2512.03503/) (2025)
- [PAPO: Stabilizing Rubric Integration Training via Decoupled Advantage Normalization](https://mlsanity.com/paper/2603.26535/) (2026)
- [IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL](https://mlsanity.com/paper/2603.12151/) (2026)
- [Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training](https://mlsanity.com/paper/2605.09608/) (2026)
- [M-GRPO: Stabilizing Self-Supervised Reinforcement Learning for Large Language Models with Momentum-Anchored Policy Optimization](https://mlsanity.com/paper/2512.13070/) (2025)
- [LiveSearchBench: An Automatically Constructed Benchmark for Retrieval and Reasoning over Dynamic Knowledge](https://mlsanity.com/paper/2511.01409/) (2025)
- [mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT](https://mlsanity.com/paper/2603.21606/) (2026)
- [One Sample to Rule Them All: Extreme Data Efficiency in Multidiscipline Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2601.03111/) (2026)
- [Where Should RL Post-Training Compute Go? Model Size, Search, Learning, and Feedback](https://mlsanity.com/paper/2607.13389/) (2026)
- [Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key](https://mlsanity.com/paper/2605.06638/) (2026)
- [A Primer in Post-Training Reasoning Data: What We Know About How It Works](https://mlsanity.com/paper/2606.02113/) (2026)
- [RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents](https://mlsanity.com/paper/2609.34712/) (2026)
- [Secure and Energy-Efficient Wireless Agentic AI Networks](https://mlsanity.com/paper/2602.15212/) (2026)
- [Continued AI Scaling Requires Repeated Efficiency Doublings](https://mlsanity.com/paper/2603.28507/) (2026)
- [Agentic Frameworks for Reasoning Tasks: An Empirical Study](https://mlsanity.com/paper/2604.16646/) (2026)
- [Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes](https://mlsanity.com/paper/2604.18381/) (2026)
- [Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors](https://mlsanity.com/paper/2606.25527/) (2026)
- [Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL](https://mlsanity.com/paper/2607.01490/) (2026)
- [Preference Tuning as Spectral Update Reorganization](https://mlsanity.com/paper/2607.20438/) (2026)
- [Performance Foundations of Parallel & Distributed Reasoning Language Models](https://mlsanity.com/paper/2608.27046/) (2026)
- [Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models](https://mlsanity.com/paper/2609.27166/) (2026)
- [Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors](https://mlsanity.com/paper/2609.34344/) (2026)
- [Question-Specific Knowledge Graphs for Efficient Visual Reasoning](https://mlsanity.com/paper/2609.35942/) (2026)
- [How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws](https://mlsanity.com/paper/2610.04158/) (2026)
- [Do Small Language Models Learn to Negotiate? A Controlled Scaling Study of RL-Trained Sellers](https://mlsanity.com/paper/2610.06204/) (2026)
- [Efficient Resource-Shiftable RL Post-training with Generative Rewards](https://mlsanity.com/paper/2610.16866/) (2026)
- [TSRL: Trigger-based Supervised Reinforcement Learning for Continual Agent Training](https://mlsanity.com/paper/2610.17726/) (2026)

## Abstract (arXiv)

While scaling laws for large language models (LLMs) during pre-training have been extensively studied, their behavior under reinforcement learning (RL) post-training remains largely unexplored. This paper presents a systematic empirical investigation of scaling behaviors in RL-based post-training, with a particular focus on mathematical reasoning. Based on a set of experiments across the full Qwen2.5 dense model series (0.5B to 72B), we characterize how model scale, data volume, and computational budget interact to shape performance. Our analysis leads to four key findings: 1. Larger models consistently exhibit superior learning efficiency on both compute and data metrics. 2. The relationship between test loss, compute, and data can be modeled by a predictive power-law which is robust across both base and instruction-tuned models. 3. Although larger models exhibit higher learning efficiency, the analytical learning efficiency term k(N) in the power-law reveals a latent saturation trend in learning efficiency as model size continues to increase. 4. In data-constrained regimes, repeated reuse of high-quality data proves highly effective, as final performance is primarily governed by the total number of optimization steps rather than the uniqueness of samples. Collectively, these results provide a principled foundation and practical guidelines for efficiently scaling the reasoning capabilities of LLMs through RL post-training.