mlsanityData to 7 Oct 2026

GRPO, Dr. GRPO, and DAPO Are Three Operations on One Number: The Group-Standard-Deviation Identity

arXiv 2607.00152 ↗First posted 30 Jun 2026cs.LG cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Three of the most popular methods for training language models to reason look like three different tricks. They are not. All three adjust a single number: standard deviation, reflecting how much a prompt's sampled answers disagree. When such a model is trained, it answers each problem many times, and an automatic checker marks every answer right or wrong. The standard deviation of those marks measures the disagreement: largest when the answers split evenly between right and wrong, and zero when they all agree. Group Relative Policy Optimization (GRPO) divides by this number, GRPO Done Right (Dr. GRPO) drops the division, and Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO) discards the groups where it is zero. Each is presented as its own fix, yet this paper proves they are three settings of one dial. That dial is not cosmetic: for right-or-wrong rewards, the disagreement is exactly the size of the training update, the group-standard-deviation identity. A split group teaches the most, while a unanimous group teaches nothing and falls silent. The same result says which problems deserve the most weight and how many tries each one needs. This paper confirms the intuition on a large real difficulty dataset (Big-Math) and in a controlled training run. What looks like a harmless normalization step is the dial that decides where learning happens and how strongly.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 14 of its 21 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

14 of 21 references.

References · 14

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  5. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 104
  6. 2025It Takes Two: Your GRPO Is Secretly DPOcited by 25
  7. 2025GRPO is Secretly a Process Reward Modelcited by 10
  8. 2025Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradientscited by 10
  9. 2024Machine Learning vs Deep Learning: The Generalization Problemcited by 6
  10. 2026When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scalingcited by 6
  11. 2026Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradientscited by 4
  12. 2026Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Workscited by 3
2 more
  1. 2026Solve for the Hyperparameter, Skip the Search: Kolmogorov-Optimal Scaling Laws for Spline Regressioncited by 2
  2. 2026A First-Principles Derivation of LLM Policy Optimization: From Expected Reward to GRPO and Its Structural Extensionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Workscited by 3
  2. 2026Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
  2. 2026Learning from Hard Prompts: Difficulty-aware Advantage Amplification in Dynamic Sampling
  3. 2026Sample What You Say: Aligning Language Models to Sample the Distributions They State
  4. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?
  5. 2026When Right Meets Wrong: Bilateral Context Conditioning with Reward-Confidence Correction for GRPO
  6. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
  7. 2026It Takes One to Bias Them All: Breaking Bad with One-Shot GRPO
  8. 2026Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems