mlsanityData to 7 Oct 2026

Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewards

arXiv 2602.11128 ↗First posted 11 Feb 2026cs.LGcited by 3

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards has driven recent advances in LLM post-training, in particular for reasoning. Policy optimization algorithms generate a number of responses for a given prompt and then effectively weight the corresponding gradients depending on the rewards. The most popular algorithms including GRPO, DAPO, and RLOO focus on ambiguous prompts, i.e., prompts with intermediate success probability, while downgrading gradients with very easy and very hard prompts. In this paper, we consider asymmetric prompt weightings that assign higher weights to prompts with low, or even zero, empirical success probability. We find that asymmetric weighting particularly benefits from-scratch RL (as in R1-Zero), where training traverses a wide accuracy range, and less so in post-SFT RL where the model already starts at high accuracy. We also provide theory that characterizes prompt weights which minimize the time needed to raise success probability from an initial level to a target accuracy under a fixed update budget. In low-success regimes, where informative responses are rare and response cost dominates, these optimal weights become asymmetric, upweighting low success probabilities and thereby accelerating effective-time convergence.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 28 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

21 of 28 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradients27 Oct
  2. 2025What is the objective of reasoning with reinforcement learning?15 Oct
  3. 2025Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting9 Oct
  4. 2025No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping26 Sep
  5. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models14 Aug
  6. 2025APO: Enhancing Reasoning Ability of MLLMs via Asymmetric Policy Optimization26 Jun
  7. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  10. 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification9 Mar
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  12. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
1 more
  1. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 8

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2025Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMscited by 61
  5. 2025JustRL: Scaling a 1.5B LLM with a Simple RL Recipecited by 57
  6. 2020Estimating Gradients for Discrete Random Variables by Sampling without Replacementcited by 29
  7. 2025What is the Alignment Objective of GRPO?cited by 23
  8. 2025When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RLcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thoughtcited by 2
  2. 2026Multimodal Reward Hacking in Reinforcement Learningcited by 2
  3. 2026HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Shrinking the Variance: Shrinkage Baselines for Reinforcement Learning with Verifiable Rewards
  2. 2026Beyond Variance: Prompt-Efficient RLVR via Rare-Event Amplification and Bidirectional Pairing
  3. 2026CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoning
  4. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
  5. 2026Prompt replay: speeding up grpo with on-policy reuse of high-signal prompts
  6. 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
  7. 2026Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization
  8. 2026Rewards as Labels: Revisiting RLVR from a Classification Perspective