mlsanityData to 14 Oct 2026

Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Model

arXiv 2603.25184 ↗First posted 26 Mar 2026cs.LG cs.AIcited by 10

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning (RL) has become essential for post-training large language models (LLMs) in reasoning tasks. While scaling rollouts can stabilize training and enhance performance, the computational overhead is a critical issue. In algorithms like GRPO, multiple rollouts per prompt incur prohibitive costs, as a large portion of prompts provide negligible gradients and are thus of low utility. To address this problem, we investigate how to select high-utility prompts before the rollout phase. Our experimental analysis reveals that sample utility is non-uniform and evolving: the strongest learning signals concentrate at the ``learning edge", the intersection of intermediate difficulty and high uncertainty, which shifts as training proceeds. Motivated by this, we propose HIVE (History-Informed and online-VErified prompt selection), a dual-stage framework for data-efficient RL. HIVE utilizes historical reward trajectories for coarse selection and employs prompt entropy as a real-time proxy to prune instances with stale utility. By evaluating HIVE across multiple math reasoning benchmarks and models, we show that HIVE yields significant rollout efficiency without compromising performance.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 77 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

29 of 77 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Learning Dynamics in RL Post-Training for Language Models8 Jan
  2. 2025LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoning1 Oct
  3. 2025MAPO: Mixed Advantage Policy Optimization23 Sep
  4. 2025RLEP: Reinforcement Learning with Experience Replay for LLM Reasoning10 Jul
  5. 2025RePO: Replay-Enhanced Policy Optimization11 Jun
  6. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  7. 2025Self-Evolving Curriculum for LLM Reasoning20 May
  8. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM19 Apr
  9. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learning7 Apr
  10. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
  11. 2025What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret3 Mar
  12. 2025LIMR: Less is More for RL Scaling17 Feb
1 more
  1. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan

Other references · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2024OpenAI o1 System Cardcited by 1,244
  7. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  8. 2025Group Sequence Policy Optimizationcited by 835
  9. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  10. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 303
  11. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 186
  12. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
4 more
  1. 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 108
  2. 2025Concise Reasoning via Reinforcement Learningcited by 79
  3. 2025Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Trainingcited by 29
  4. 2026No More Stale Feedback: Co-Evolving Critics for Open-World Agent Learningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Policy and World Modeling Co-Training for Language Agentscited by 15
  2. 2026AHD Agent: Agentic Reinforcement Learning for Automatic Heuristic Designcited by 3
  3. 2026GD2PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimizationcited by 3
  4. 2026Beyond Solvability: Task Learnability as a Static Prior for LLM RL Post-Trainingcited by 2
  5. 2026Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Trainingcited by 1
  6. 2026Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agentscited by 1
  7. 2026Gradient Extrapolation-Based Policy Optimizationcited by 0
  8. 2026AdaKP: Online Adaptive Knowledge-Point Selection for Reasoning-Oriented Reinforcement Learningcited by 0
  9. 2026Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directionscited by 0
  10. 2026Selecting Diverse SFT Traces Improves Post-RL Generalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models
  2. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
  3. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learning
  4. 2025Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models?
  5. 2025Accelerating RL for LLM Reasoning with Optimal Advantage Regression
  6. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  7. 2026Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models
  8. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement