mlsanityData to 14 Oct 2026

G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance

arXiv 2508.13023 ↗First posted 18 Aug 2025cs.AIcited by 10

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has markedly enhanced the reasoning abilities of large language models (LLMs). Its success, however, largely depends on strong base models with rich world knowledge, yielding only modest improvements for small-size language models (SLMs). To address this limitation, we investigate Guided GRPO, which injects ground-truth reasoning steps into roll-out trajectories to compensate for SLMs' inherent weaknesses. Through a comprehensive study of various guidance configurations, we find that naively adding guidance delivers limited gains. These insights motivate G2RPO-A, an adaptive algorithm that automatically adjusts guidance strength in response to the model's evolving training dynamics. Experiments on mathematical reasoning and code-generation benchmarks confirm that G2RPO-A substantially outperforms vanilla GRPO. Our code and models are available at https://github.com/T-Lab-CUHKSZ/G2RPO-A.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 41 of its 50 references that are in the map. First posted on arXiv 18 Aug 2025.

Builds on

41 of 50 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun
  2. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO9 Jun
  3. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning7 Jun
  4. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts2 Jun
  5. 2025DISCO Balances the Scales: Adaptive Domain- and Difficulty-Aware Reinforcement Learning on Imbalanced Data21 May
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  7. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce15 Apr
  8. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learning7 Apr
  9. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning4 Apr
  10. 2025Boosting MLLM Reasoning with Text-Debiased Hint-GRPO31 Mar
  11. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  12. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't20 Mar
7 more
  1. 2025Adaptive Group Policy Optimization: Towards Stable Training and Token-Efficient Reasoning20 Mar
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  3. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization17 Mar
  4. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution25 Feb
  5. 2025Process Reinforcement through Implicit Rewards3 Feb
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  7. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 22

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  10. 2025s1: Simple test-time scalingcited by 1,341
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  12. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
10 more
  1. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  2. 2025LIMO: Less is More for Reasoningcited by 498
  3. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  4. 2025ToRL: Scaling Tool-Integrated RLcited by 169
  5. 2024Small Language Models: Survey, Measurements, and Insightscited by 112
  6. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Modelscited by 94
  7. 2025Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewardscited by 50
  8. 2025Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Mathcited by 44
  9. 2024EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuningcited by 17
  10. 2025Effective Learning for Small Reasoning Models: An Empirical Study on 0.5B Reasoning LLMscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 9
  3. 2026KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidancecited by 8
  4. 2026Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Informationcited by 7
  5. 2026Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimizationcited by 4
  6. 2025APO: Alpha-Divergence Preference Optimizationcited by 2
  7. 2026FedMOA: Federated GRPO for Personalized Reasoning LLMs under Heterogeneous Rewardscited by 2
  8. 2026Beyond KL Divergence: Policy Optimization with Flexible Bregman Divergences for LLM Reasoningcited by 2
  9. 2025Multi-Reward GRPO Fine-Tuning for De-biasing Large Language Models: A Study Based on Chinese-Context Discrimination Datacited by 0
  10. 2026Learning Structured Reasoning via Tractable Trajectory Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Sharpness-Guided Group Relative Policy Optimization via Probability Shaping
  2. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
  3. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models
  4. 2026GRPO Training Dynamics for Small Language Models
  5. 2026GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning
  6. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?
  7. 2026CoDistill-GRPO: A Co-Distillation Recipe for Efficient Group Relative Policy Optimization
  8. 2025Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning