mlsanityData to 7 Oct 2026

CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoning

arXiv 2605.24331 ↗First posted 23 May 2026cs.LG stat.MLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Context or prompt-level reweighting has emerged as a central algorithmic lever in Reinforcement Learning with Verified Rewards (RLVR) for improving the reasoning capability of large language models, yet the principle determining what constitutes an optimal weighting remains poorly understood. We address this gap by formulating prompt reweighting as a functional derivative of a utility functional defined in the pass-rate function space, yielding a unified optimality framework that accommodates existing schemes, including REINFORCE and GRPO. Building on this optimality framework, we propose a distribution-aware prompt reweighting approach, called CurveRL, based on a quantile coordinate transform, in which the weight assigned to each prompt depends not on the absolute value of pass rates but on its rank and density to reflect the distributional structure of the pass rates in the learning dynamics. Extensive experiments across multiple benchmarks demonstrate that our proposed CurveRL consistently outperforms GRPO and other RLVR baselines. Our study identifies context-distribution control as a principled axis for analyzing and designing prompt-reweighted RLVR algorithms. The code is released in https://github.com/zhyzmath/CurveRL.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 110 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

38 of 110 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum18 Mar
  2. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic1 Mar
  3. 2026Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training24 Feb
  4. 2026On the Emergence of Implicit Curriculum in RLVR Learning Dynamics16 Feb
  5. 2026Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients30 Jan
  6. 2026Your Group-Relative Advantage Is Biased13 Jan
  7. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models8 Dec
  8. 2025Beyond High-Entropy Exploration: Correctness-Aware Low-Entropy Segment-Based Advantage Shaping for Reasoning LLMs30 Nov
  9. 2025What is the objective of reasoning with reinforcement learning?15 Oct
  10. 2025Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning13 Oct
  11. 2025Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectives6 Oct
  12. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models14 Aug
6 more
  1. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning7 Jun
  2. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  3. 2025Self-Evolving Curriculum for LLM Reasoning20 May
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification9 Mar
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan

Other references · 20

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2024OpenAI o1 System Cardcited by 1,220
  7. 2015Prioritized Experience Replaycited by 1,070
  8. 2025Group Sequence Policy Optimizationcited by 788
  9. 2018A Tutorial on Bayesian Optimizationcited by 734
  10. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  11. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  12. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 159
8 more
  1. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  2. 2025Olmo 3cited by 95
  3. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origincited by 92
  4. 2026Maximum Likelihood Reinforcement Learningcited by 29
  5. 2024Dual Active Learning for Reinforcement Learning from Human Feedbackcited by 19
  6. 2015Emphatic Temporal-Difference Learningcited by 16
  7. 2018Simple Regret Minimization for Contextual Banditscited by 16
  8. 2026Multiplex Thinking: Reasoning via Token-wise Branch-and-Mergecited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewards
  2. 2026Re2: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
  3. 2025FlowRL: Matching Reward Distributions for LLM Reasoning
  4. 2025From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization
  5. 2026Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
  6. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  7. 2026Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
  8. 2025No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping