mlsanityData to 7 Oct 2026

Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO

arXiv 2609.38018 ↗First posted 29 Sep 2026cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Group relative policy optimization (GRPO) learns only from prompts whose sampled responses disagree: a group that is entirely correct or entirely incorrect has zero reward variance, contributes no gradient, and still consumes its rollouts. Prompt-selection methods reduce this waste by steering sampling toward intermediate pass rates, but they choose the target, its width, and the uncertainty model heuristically, in raw pass-rate or logit coordinates. We show that GRPO comes with a natural coordinate for pass rates: the arc length psi=arcsin√p on the Bernoulli Fisher--Rao manifold. In arc length, the expected GRPO update is uniform up to two boundary ramps; the probability of a zero-variance group is bounded by two Gaussian boundary layers of width 1/√2G; pass-rate evidence has constant noise; and the gradients of the pass@k and passk objectives are Gaussians whose center and width follow from k in closed form. A prompt curriculum for GRPO is therefore a Gaussian in arc length, and choosing its center amounts to choosing the objective. We turn this observation into ARCUS, a drop-in sampler that tracks every prompt with a Kalman filter in arc length, scores prompts by an objective-matched Gaussian kernel times the predicted probability of an informative group, keeps only informative groups for the unchanged GRPO update, and paces the target toward the hardest objective whose predicted yield stays within a small slack of the best. Across six mathematical reasoning benchmarks and three backbones, ARCUS improves the average accuracy of GRPO by 2.8--2.9 points and that of dynamic sampling by 1.1--1.2 points, while generating 48--57% fewer rollouts than dynamic sampling.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 79 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

48 of 79 references.

Closest ancestors · 35

Same direction, newest first

  1. 2026ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVR8 Sep
  2. 2026LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models30 Jul
  3. 2026Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning30 Jul
  4. 2026Learning as Reasoning Unfolds: Progressive Rollout Allocation for Efficient Reinforcement Learning24 Jul
  5. 2026Effective Reinforcement Learning for Agentic Search by Recycling Zero-Variance Queries During Training9 Jun
  6. 2026RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning31 May
  7. 2026fg-expo: Frontier-guided exploration-prioritized policy optimization via adaptive kl and gaussian curriculum12 May
  8. 2026Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR8 May
  9. 2026Prompt replay: speeding up grpo with on-policy reuse of high-signal prompts22 Mar
  10. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic1 Mar
  11. 2026Beyond Variance: Prompt-Efficient RLVR via Rare-Event Amplification and Bidirectional Pairing3 Feb
  12. 2026Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models2 Feb
23 more
  1. 2025Each Prompt Matters: Scaling Reinforcement Learning Without Wasting Rollouts on Hundred-Billion-Scale MoE8 Dec
  2. 2025Explore Data Left Behind in Reinforcement Learning for Reasoning Language Models6 Nov
  3. 2025What is the objective of reasoning with reinforcement learning?15 Oct
  4. 2025CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs1 Oct
  5. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models24 Sep
  6. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models14 Aug
  7. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  8. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning7 Jun
  9. 2025BNPO: Beta Normalization Policy Optimization3 Jun
  10. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts2 Jun
  11. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models30 May
  12. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  13. 2025Self-Evolving Curriculum for LLM Reasoning20 May
  14. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  15. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  16. 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification9 Mar
  17. 2025Learning to Reason at the Frontier of Learnability17 Feb
  18. 2025LIMR: Less is More for RL Scaling17 Feb
  19. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  20. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  21. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan
  22. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  23. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  7. 2025Group Sequence Policy Optimizationcited by 788
  8. 2025Skywork Open Reasoner 1 Technical Reportcited by 161
  9. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 83
  10. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
  11. 2025Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocationcited by 21
  12. 2026TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learningcited by 9
1 more
  1. 2026DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewardscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Composing Policy Gradients and Prompt Optimization for Language Model Programs
  2. 2026Constrained Group Relative Policy Optimization
  3. 2025Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training
  4. 2026How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
  5. 2025Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environments
  6. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
  7. 2026On the Impossibility of Unbiased and Length-Invariant Policy Optimization with Outcome Rewards
  8. 2026MC-GRPO: Median-Centered Group Relative Policy Optimization for Small-Rollout Reinforcement Learning