mlsanityData to 14 Oct 2026

Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance

arXiv 2605.15012 ↗First posted 14 May 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has achieved great success in developing Large Language Models (LLMs) with chain-of-thought rollouts for many tasks such as math and coding. Nevertheless, RLVR struggles with sample efficiency on difficult problems where correct rollouts are hard to generate. Prior works propose to address this issue via demonstration-guided RLVR, i.e., to conduct Supervised FineTuning (SFT) when RL fails; however, SFT often requires a lot of data, which can be expensive to acquire. In this paper, we propose FEST, a FEw-ShoT demonstration-guided RLVR algorithm. It attains compelling results with only 128 demonstrations randomly selected from an SFT dataset. We find that three components are vital for the success: supervised signal, on-policy signal, and decaying weights on the few-shot SFT dataset to prevent overfitting from multiple-epoch training. On several benchmarks, FEST outperforms baselines with magnitudes less SFT data, even matching their performance with full dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 90 of its 108 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

90 of 108 references.

Closest ancestors · 25

Same direction, newest first

  1. 2025TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoning15 Dec
  2. 2025DRIVE: Data Curation Best Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generation9 Nov
  3. 2025Mitigating Forgetting Between Supervised and Reinforcement Learning Yields Stronger Reasoners6 Oct
  4. 2025It Takes Two: Your GRPO Is Secretly DPO1 Oct
  5. 2025Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse30 Sep
  6. 2025Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling2 Jul
  7. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning24 Jun
  8. 2025Reasoning with Exploration: An Entropy Perspective17 Jun
  9. 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions9 Jun
  10. 2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models5 Jun
  11. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning2 Jun
  12. 2025SuperRL: Reinforcement Learning with Supervision to Boost Language Model Reasoning1 Jun
13 more
  1. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data25 May
  2. 2025UFT: Unifying Supervised and Reinforcement Fine-Tuning22 May
  3. 2025Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs19 May
  4. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  5. 2025Learning to Reason under Off-Policy Guidance21 Apr
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  7. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining10 Apr
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  10. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models24 Feb
  11. 2025LIMR: Less is More for RL Scaling17 Feb
  12. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  13. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 65

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  10. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  12. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
38 more
  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  4. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  5. 2025s1: Simple test-time scalingcited by 1,341
  6. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  8. 2025Group Sequence Policy Optimizationcited by 835
  9. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
  10. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  11. 2025LIMO: Less is More for Reasoningcited by 498
  12. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 350
  13. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 263
  14. 2024Direct Language Model Alignment from Online AI Feedbackcited by 260
  15. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 241
  16. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  17. 2024Secrets of RLHF in Large Language Models Part II: Reward Modelingcited by 171
  18. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 164
  19. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  20. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origincited by 97
  21. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 91
  22. 2025Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipelinecited by 82
  23. 2023Data Diversity Matters for Robust Instruction Tuningcited by 76
  24. 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
  25. 2025Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoningcited by 68
  26. 2025Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMscited by 61
  27. 2023Instruction Mining: Instruction Data Selection for Tuning Large Language Modelscited by 58
  28. 2025The Best Instruction-Tuning Data are Those That Fitcited by 52
  29. 2024Preference Learning Algorithms Do Not Learn Preference Rankingscited by 48
  30. 2024Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignmentcited by 46
  31. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 43
  32. 2025Towards a Unified View of Large Language Model Post-Trainingcited by 36
  33. 2025Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluationcited by 29
  34. 2025Online Preference Alignment for Language Models via Count-based Explorationcited by 26
  35. 2025Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metriccited by 24
  36. 2025CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignmentcited by 24
  37. 2023Scaling Down to Scale Up: A Cost-Benefit Analysis of Replacing OpenAI's LLM with Open Source SLMs in Productioncited by 21
  38. 2025Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Modelscited by 19

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  2. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
  3. 2026When Self-Belief Misleads: Active Label Acquisition for Reinforcement Learning with Verifiable Rewards
  4. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  5. 2025On the optimization dynamics of RLVR: Gradient gap and step size thresholds
  6. 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewards
  7. 2026You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories
  8. 2026From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement