mlsanityData to 14 Oct 2026

LIMR: Less is More for RL Scaling

arXiv 2502.11886 ↗First posted 17 Feb 2025cs.LG cs.AI cs.CLcited by 139

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

In this paper, we ask: what truly determines the effectiveness of RL training data for enhancing language models' reasoning capabilities? While recent advances like o1, Deepseek R1, and Kimi1.5 demonstrate RL's potential, the lack of transparency about training data requirements has hindered systematic progress. Starting directly from base models without distillation, we challenge the assumption that scaling up RL training data inherently improves performance. we demonstrate that a strategically selected subset of just 1,389 samples can outperform the full 8,523-sample dataset. We introduce Learning Impact Measurement (LIM), an automated method to evaluate and prioritize training samples based on their alignment with model learning trajectories, enabling efficient resource utilization and scalable implementation. Our method achieves comparable or even superior performance using only 1,389 samples versus the full 8,523 samples dataset. Notably, while recent data-efficient approaches (e.g., LIMO and s1) show promise with 32B-scale models, we find it significantly underperforms at 7B-scale through supervised fine-tuning (SFT). In contrast, our RL-based LIMR achieves 16.7% higher accuracy on AIME24 and outperforms LIMO and s1 by 13.0% and 22.2% on MATH500. These results fundamentally reshape our understanding of RL scaling in LLMs, demonstrating that precise sample selection, rather than data scale, may be the key to unlocking enhanced reasoning capabilities. For reproducible research and future innovation, we are open-sourcing LIMR, including implementation of LIM, training and evaluation code, curated datasets, and trained models at https://github.com/GAIR-NLP/LIMR.

Key facts. Cited by 139 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 15 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

6 of 15 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 5

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  4. 2025LIMO: Less is More for Reasoningcited by 498
  5. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scalingcited by 77

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 85; 2026: 54.

139 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  3. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  4. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  5. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  6. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  7. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  8. 2025Hierarchical Reasoning Modelcited by 177
  9. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  10. 2025ToRL: Scaling Tool-Integrated RLcited by 169
  11. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 162
  12. 2025Phi-4-reasoning Technical Reportcited by 151
38 more
  1. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 113
  2. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 104
  3. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 93
  4. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 93
  5. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 81
  6. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 68
  7. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 56
  8. 2025Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models?cited by 52
  9. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 42
  10. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 41
  11. 2025The Hallucination Tax of Reinforcement Finetuningcited by 40
  12. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resourcescited by 39
  13. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 36
  14. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  15. 2025Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoningcited by 30
  16. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learningcited by 29
  17. 2025Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoningcited by 29
  18. 2025Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedbackcited by 26
  19. 2025VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domainscited by 24
  20. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 24
  21. 2025HardTests: Synthesizing High-Quality Test Cases for LLM Codingcited by 23
  22. 2025OptimAI: Optimization from Natural Language Using LLM-Powered AI Agentscited by 22
  23. 2025VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learningcited by 21
  24. 2025Revisiting Entropy in Reinforcement Learning for Large Reasoning Modelscited by 21
  25. 2025Towards High Data Efficiency in Reinforcement Learning with Verifiable Rewardcited by 19
  26. 2025XRPO: Pushing the limits of GRPO with Targeted Exploration and Exploitationcited by 18
  27. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  28. 2025Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoningcited by 15
  29. 2026Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Modelscited by 14
  30. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 12
  31. 2025SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesiscited by 12
  32. 2025Scheduling Your LLM Reinforcement Learning with Reasoning Treescited by 12
  33. 2025Data-Efficient RLVR via Off-Policy Influence Guidancecited by 12
  34. 2026GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learningcited by 12
  35. 2025A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learningcited by 11
  36. 2025Surrogate Signals from Format and Length: Reinforcement Learning for Solving Mathematical Problems without Ground Truth Answerscited by 11
  37. 2026Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Modelscited by 11
  38. 2025Reinforcement Learning Teachers of Test Time Scalingcited by 10

50 most cited of 139 All 139 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Models
  2. 2026PRISM: Demystifying Retention and Interaction in Mid-Training
  3. 2025Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
  4. 2025Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuning
  5. 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
  6. 2025Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levels
  7. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  8. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training