mlsanityData to 14 Oct 2026

ScRPO: From Errors to Insights

arXiv 2511.06065 ↗First posted 8 Nov 2025cs.AI cs.CLcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

We introduce Self-correction Relative Policy Optimization (ScRPO), a novel reinforcement learning framework designed to empower large language models with advanced mathematical reasoning capabilities through iterative self-reflection and error correction. The ScRPO framework operates in two distinct phases: (1) Trial-and-error learning stage, where the model is trained via GRPO, and incorrect responses are collected to form an "error pool"; and (2) Self-correction learning stage, which guides the model to introspectively analyze and rectify the reasoning flaws behind its previous errors. Extensive evaluations across challenging mathematical benchmarks, including AIME, AMC, Olympiad, MATH-500, and GSM8k, validate the efficacy of our approach. Using DeepSeek-R1-Distill-Qwen-1.5B and 7B as backbones, ScRPO achieves average accuracies of 64.8% and 77.8%, respectively. This represents a significant improvement of 6.0% and 3.2% over vanilla baselines, consistently outperforming strong post-training methods such as DAPO and GRPO. These findings establish ScRPO as a robust paradigm for enabling autonomous self-improvement in AI systems, particularly in tasks with limited external feedback.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 32 references that are in the map. First posted on arXiv 8 Nov 2025.

Builds on

21 of 32 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  2. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs3 Mar
  3. 2025Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuning10 Feb
  4. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search4 Feb
  5. 2023Let's Verify Step by Step31 May
  6. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 15

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  7. 2023Learning From Mistakes Makes LLM Better Reasonercited by 101
  8. 2025Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitatecited by 63
  9. 2024AI Flow at the Network Edgecited by 46
  10. 2025AI Flow: Perspectives, Scenarios, and Approachescited by 44
  11. 2025RLVR-World: Training World Models with Reinforcement Learningcited by 40
  12. 2025Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learningcited by 33
3 more
  1. 2025Hybrid Group Relative Policy Optimization: A Multi-Sample Approach to Enhancing Policy Optimizationcited by 15
  2. 2025Structural Reward Model: Enhancing Interpretability, Efficiency, and Scalability in Reward Modelingcited by 11
  3. 2025Advancing Multi-Step Mathematical Reasoning in Large Language Models through Multi-Layered Self-Reflection with Auto-Promptingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Skip-Connected Policy Optimization for Implicit Advantagecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning
  2. 2025Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning
  3. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM
  4. 2025Multi-Layer GRPO: Enhancing Reasoning and Self-Correction in Large Language Models
  5. 2026iGRPO: Self-Feedback-Driven LLM Reasoning
  6. 2026Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
  7. 2025CoRPO: Adding a Correctness Bias to GRPO Improves Generalization
  8. 2026DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning