mlsanityData to 14 Oct 2026

CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents

arXiv 2605.25511 ↗First posted 25 May 2026cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Recent advancements in Reinforcement Learning (RL), particularly Group Relative Policy Optimization (GRPO), have significantly enhanced the reasoning capabilities of Large Language Models. However, applying these problem-centric optimization methods to role-playing agents often leads to a loss of character fidelity and style collapse, as they prioritize context-specific utility over persona alignment. To address this, we propose Character-Centric Group Relative Policy Optimization (CRPO), a framework designed to realign RL objectives with the role-playing task. CRPO improves character distinctiveness through three mechanisms: decoupling task logic from stylistic rewards to resolve gradient conflicts, dynamically adapting optimization constraints based on character complexity, and utilizing generic responses as negative baselines to prevent the model from reverting to a common distribution. Extensive experiments demonstrate that CRPO outperforms existing methods in consistency, emotion and others.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 37 of its 69 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

37 of 69 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning12 Jan
  2. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization8 Jan
  3. 2026AMIR-GRPO: Inducing Implicit Preference Signals into GRPO7 Jan
  4. 2025DaGRPO: Rectifying Gradient Conflict in Reasoning via Distinctiveness-Aware Group Relative Policy Optimization6 Dec
  5. 2025Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning22 Oct
  6. 2025Training-Free Group Relative Policy Optimization9 Oct
  7. 2025SCOPE-RL: Stable and Quantitative Control of Policy Entropy in RL Post-Training9 Oct
  8. 2025λ-GRPO: Unifying the GRPO Frameworks with Learnable Token Preferences8 Oct
  9. 2025Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning5 Oct
  10. 2025Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoning2 Oct
  11. 2025NGRPO: Negative-enhanced Group Relative Policy Optimization23 Sep
  12. 2025GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control5 Aug
6 more
  1. 2025EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity29 Jul
  2. 2025Kalman Filter Enhanced GRPO for Reinforcement Learning-Based Language Model Reasoning12 May
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan
  6. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 19

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  5. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  6. 2025Group Sequence Policy Optimizationcited by 835
  7. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  8. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 367
  9. 2025RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agentscited by 25
  10. 2025Sotopia-RL: Reward Design for Social Intelligencecited by 24
  11. 2025RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Rewardcited by 13
  12. 2025MOOM: Maintenance, Organization and Optimization of Memory in Ultra-Long Role-Playing Dialoguescited by 8
7 more
  1. 2025HyCoRA: Hyper-Contrastive Role-Adaptive Learning for Role-Playingcited by 7
  2. 2024Thinking Before Speaking: A Role-playing Model with Mindsetcited by 5
  3. 2025MOA: Multi-Objective Alignment for Role-Playing Agentscited by 5
  4. 2026Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVRcited by 5
  5. 2026Deriving Character Logic from Storyline as Codified Decision Treescited by 3
  6. 2025SGPO: Self-Generated Preference Optimization based on Self-Improvercited by 1
  7. 2025Understanding Generalization in Role-Playing Models via Information Theorycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoning
  2. 2026Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
  3. 2025CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization
  4. 2026MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Models
  5. 2026Adversarial Closed-Loop Curriculum for Evolving Role-Playing Agents
  6. 2025CoRPO: Adding a Correctness Bias to GRPO Improves Generalization
  7. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
  8. 2026Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective