mlsanityData to 7 Oct 2026

GCPO: When Contrast Fails, Go Gold

arXiv 2510.07790 ↗First posted 9 Oct 2025cs.AIcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning has been widely applied to enhance the reasoning capabilities of large language models. Extending the inference limits of smaller models has become a prominent research focus. However, algorithms such as Group Relative Policy Optimization (GRPO) suffer from a clear drawback: the upper bound of a model's rollout responses is entirely determined by the model itself, preventing the acquisition of knowledge from samples that are either all incorrect or all correct. In this paper, we introduce Group Contrastive Policy Optimization (GCPO), a method that incorporates external standard reference answers. When the model cannot solve a problem, the reference answer supplies the correct response, steering the model toward an unequivocally accurate update direction. This approach offers two main advantages: (1) it improves training efficiency by fully utilizing every sample; (2) it enables the model to emulate the problem solving strategy of the reference answer during training, thereby enhancing generalization in reasoning. GCPO achieves outstanding results across multiple benchmark datasets, yielding substantial improvements over the baseline model. Our code is available at: https://github.com/AchoWu/GCPO.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 50 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

46 of 50 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  2. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening3 Jun
  3. 2025Maximizing Confidence Alone Improves Reasoning28 May
  4. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  5. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM19 Apr
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  7. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  8. 2025Boosting MLLM Reasoning with Text-Debiased Hint-GRPO31 Mar
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  10. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  11. 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification9 Mar
  12. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
4 more
  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  2. 2024Group Robust Preference Optimization in Reward-free RLHF30 May
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  4. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  11. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  12. 2020Knowledge Distillation: A Surveycited by 1,578
18 more
  1. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  2. 2025s1: Simple test-time scalingcited by 1,311
  3. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  4. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  5. 2025Group Sequence Policy Optimizationcited by 788
  6. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  7. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  8. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  9. 2022Large Language Models Are Reasoning Teacherscited by 318
  10. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  11. 2022Teaching Small Language Models to Reasoncited by 279
  12. 2024Reinforcement learningcited by 236
  13. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  14. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  15. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?cited by 153
  16. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approachescited by 43
  17. 2023MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Promptingcited by 27
  18. 2024Improving Reinforcement Learning from Human Feedback Using Contrastive Rewardscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026On the Hidden Objective Biases of Group-based Reinforcement Learningcited by 2
  2. 2026S-GRPO: Unified Post-Training for Large Vision-Language Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
  2. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
  3. 2025CoRPO: Adding a Correctness Bias to GRPO Improves Generalization
  4. 2025Group Causal Policy Optimization for Post-Training Large Language Models
  5. 2025Kalman Filter Enhanced GRPO for Reinforcement Learning-Based Language Model Reasoning
  6. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning
  7. 2025EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity
  8. 2026GRPO Training Dynamics for Small Language Models