mlsanityData to 7 Oct 2026

Beyond Variance: Prompt-Efficient RLVR via Rare-Event Amplification and Bidirectional Pairing

arXiv 2602.03452 ↗First posted 3 Feb 2026cs.LG cs.AIcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) is effective for training large language models on deterministic outcome reasoning tasks. Prior work shows RLVR works with few prompts, but prompt selection is often based only on training-accuracy variance, leading to unstable optimization directions and weaker transfer. We revisit prompt selection from a mechanism-level view and argue that an effective minibatch should provide both (i) a reliable positive anchor and (ii) explicit negative learning signals from rare failures. Based on this principle, we propose positive--negative pairing: at each update, we sample a hard-but-solvable q+ and an easy-but-brittle prompt q-(high success rate but not perfect), characterized by low and high empirical success rates under multiple rollouts. We further introduce Weighted GRPO, which reweights binary outcomes at the pair level and uses group-normalized advantages to amplify rare successes on q+ into sharp positive guidance while turning rare failures on q- into strong negative penalties. This bidirectional signal provides informative learning feedback for both successes and failures, improving sample efficiency without suppressing exploration. On Qwen2.5-Math-7B, a single paired minibatch per update consistently outperforms a GRPO baseline that selects two prompts via commonly used variance-based selection heuristics: AIME~2025 Pass@8 improves from 16.8 to 22.2, and AMC23 Pass@64 from 94.0 to 97.0, while remaining competitive with large-scale RLVR trained from a pool of 1209 training prompts. Similar gains are observed on Qwen2.5-Math-7B-Instruct.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 40 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

34 of 40 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting9 Oct
  2. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning2 Jun
  3. 2025Unearthing Gems from Stones: Policy Optimization with Negative Sample Augmentation for LLM Reasoning20 May
  4. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO16 May
  5. 2025INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning12 May
  6. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  7. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM19 Apr
  8. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  10. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  11. 2025What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret3 Mar
  12. 2025LIMR: Less is More for RL Scaling17 Feb
7 more
  1. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  2. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
  3. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  4. 2024On Designing Effective RL Reward at Training Time for LLM Reasoning19 Oct
  5. 2024Process Reward Model with Q-Value Rankings15 Oct
  6. 2023Let's Verify Step by Step31 May
  7. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 15

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  4. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  5. 2024InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modelingcited by 76
  6. 2024Active Preference Optimization for Sample Efficient RLHFcited by 61
  7. 2024Active Preference Learning for Large Language Modelscited by 55
  8. 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 40
  9. 2025Large-Scale Data Selection for Instruction Tuningcited by 34
  10. 2025Scalable Chain of Thoughts via Elastic Reasoningcited by 32
  11. 2025Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewardscited by 29
  12. 2025Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMscited by 28
3 more
  1. 2022Data-Efficient Finetuning Using Cross-Task Nearest Neighborscited by 26
  2. 2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learningcited by 20
  3. 2024Enabling Weak LLMs to Judge Response Reliability via Meta Rankingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPOcited by 0
  2. 2026Unlearnable, or Unmeasured? On the Reliability of Difficulty Labels in RLVRcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping
  2. 2026MaPP: A Unified Marginalized Posterior-Predictive Framework for Data-Efficient RLVR
  3. 2026Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewards
  4. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  5. 2025On the optimization dynamics of RLVR: Gradient gap and step size thresholds
  6. 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewards
  7. 2026LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models
  8. 2026P^2O: Joint Policy and Prompt Optimization