mlsanityData to 7 Oct 2026

Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models

arXiv 2602.01970 ↗First posted 2 Feb 2026cs.AI cs.LGcited by 11

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning enhances the reasoning capabilities of large language models but often involves high computational costs due to rollout-intensive optimization. Online prompt selection presents a plausible solution by prioritizing informative prompts to improve training efficiency. However, current methods either depend on costly, exact evaluations or construct prompt-specific predictive models lacking generalization across prompts. This study introduces Generalizable Predictive Prompt Selection (GPS), which performs Bayesian inference towards prompt difficulty using a lightweight generative model trained on the shared optimization history. Intermediate-difficulty prioritization and history-anchored diversity are incorporated into the batch acquisition principle to select informative prompt batches. The small predictive model also generalizes at test-time for efficient computational allocation. Experiments across varied reasoning benchmarks indicate GPS's substantial improvements in training efficiency, final performance, and test-time efficiency over superior baseline methods.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 66 of its 87 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

66 of 87 references.

Closest ancestors · 29

Same direction, newest first

  1. 2025Revisiting Entropy in Reinforcement Learning for Large Reasoning Models8 Nov
  2. 2025CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs1 Oct
  3. 2025Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models?7 Jul
  4. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learning10 Jun
  5. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library6 Jun
  6. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay5 Jun
  7. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts2 Jun
  8. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models30 May
  9. 2025Self-Evolving Curriculum for LLM Reasoning20 May
  10. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  11. 2025Learning to Reason under Off-Policy Guidance21 Apr
  12. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM19 Apr
17 more
  1. 2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning18 Apr
  2. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
  3. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning4 Apr
  4. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  5. 2025CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models28 Mar
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  7. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  8. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't20 Mar
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  10. 2025What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret3 Mar
  11. 2025LIMR: Less is More for RL Scaling17 Feb
  12. 2025Process Reinforcement through Implicit Rewards3 Feb
  13. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  14. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  15. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan
  16. 2024RLHF Workflow: From Reward Modeling to Online RLHF13 May
  17. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  11. 2024OpenAI o1 System Cardcited by 1,220
  12. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
25 more
  1. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  2. 2025Group Sequence Policy Optimizationcited by 788
  3. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  4. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  5. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  6. 2025LIMO: Less is More for Reasoningcited by 488
  7. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  8. 2018Neural Processescited by 307
  9. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  10. 2025Deep Think with Confidencecited by 199
  11. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  12. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 168
  13. 2025Concise Reasoning via Reinforcement Learningcited by 78
  14. 2025What Makes a Reward Model a Good Teacher? An Optimization Perspectivecited by 70
  15. 2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computationcited by 67
  16. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
  17. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  18. 2024Doubly Mild Generalization for Offline Reinforcement Learningcited by 21
  19. 2025Model Predictive Task Sampling for Efficient and Robust Adaptationcited by 20
  20. 2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppressioncited by 17
  21. 2025Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuningcited by 14
  22. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  23. 2024Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Explorationcited by 9
  24. 2025AdaBoN: Adaptive Best-of-N Alignmentcited by 8
  25. 2025OptPO: Optimal Rollout Allocation for Test-time Policy Optimizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learningcited by 9
  2. 2026Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVRcited by 5
  3. 2026Spend Your Rollouts Where It Counts: Rollout Allocation for Group-Based RL Post-Trainingcited by 5
  4. 2026Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplexcited by 2
  5. 2026Early Verdicts, Better Budgets: Sequential Adaptive Rollout Allocation for Compute-Efficient RLVRcited by 2
  6. 2026PAIR: Pairwise-Aware Inclusion Reweighting for Adaptive Rollout Allocation in RLVRcited by 1
  7. 2026ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVRcited by 1
  8. 2026You Only Edit Once: Incentivizing In-Context Capability of LLMs via Local Demonstration Refinementcited by 0
  9. 2026Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPOcited by 0
  10. 2026Mitigating the Length-Scaling Tax with Online Distillationcited by 0
  11. 2026Unlearnable, or Unmeasured? On the Reliability of Difficulty Labels in RLVRcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Model
  2. 2026Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models
  3. 2026Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning
  4. 2024GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers
  5. 2025Dynamic Policy Induction for Adaptive Prompt Optimization: Bridging the Efficiency-Accuracy Gap via Lightweight Reinforcement Learning
  6. 2025Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning
  7. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  8. 2025Prompt-Tuning Bandits: Enabling Few-Shot Generalization for Efficient Multi-Task Offline RL