# Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVR

arXiv: https://arxiv.org/abs/2601.04611 · HTML: https://mlsanity.com/paper/2601.04611/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Character-LLM: A Trainable Agent for Role-Playing](https://mlsanity.com/paper/2310.10158/) (2023)
- [Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training](https://mlsanity.com/paper/2309.17179/) (2023)
- [InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews](https://mlsanity.com/paper/2310.17976/) (2023)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!](https://mlsanity.com/paper/2502.07374/) (2025)
- [Small Models Struggle to Learn from Strong Reasoners](https://mlsanity.com/paper/2502.12143/) (2025)
- [Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models](https://mlsanity.com/paper/2304.13835/) (2023)
- [Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent](https://mlsanity.com/paper/2402.13717/) (2024)
- [The Oscars of AI Theater: A Survey on Role-Playing with Language Models](https://mlsanity.com/paper/2407.11484/) (2024)
- [Thinking Machines: A Survey of LLM based Reasoning Strategies](https://mlsanity.com/paper/2503.10814/) (2025)
- [RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents](https://mlsanity.com/paper/2507.03112/) (2025)
- [Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona](https://mlsanity.com/paper/2305.11482/) (2023)
- [Identity-Driven Hierarchical Role-Playing Agents](https://mlsanity.com/paper/2407.19412/) (2024)
- [RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following](https://mlsanity.com/paper/2502.11387/) (2025)
- [Reasoning Does Not Necessarily Improve Role-Playing Ability](https://mlsanity.com/paper/2502.16940/) (2025)
- [Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoning](https://mlsanity.com/paper/2506.01748/) (2025)
- [RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward](https://mlsanity.com/paper/2505.10218/) (2025)

## Built on by (most cited first, 5 of 5)

- [Agentic Tool Use in Large Language Models](https://mlsanity.com/paper/2604.00835/) (2026)
- [Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization](https://mlsanity.com/paper/2606.27025/) (2026)
- [CustomerSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators](https://mlsanity.com/paper/2605.08334/) (2026)
- [Through the Lens of Character: Resolving Modality-Role Interference in Multimodal Role-Playing Agent](https://mlsanity.com/paper/2605.09443/) (2026)
- [CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents](https://mlsanity.com/paper/2605.25511/) (2026)

## Abstract (arXiv)

Current role-playing agents (RPAs) are typically constructed by imitating surface-level behaviors, but this approach lacks internal cognitive consistency, often causing out-of-character errors in complex situations. To address this, we propose Character-R1, a framework designed to provide comprehensive verifiable reward signals for effective role-aware reasoning, which are missing in recent studies. Specifically, our framework comprises three core designs: (1) Cognitive Focus Reward, which enforces explicit label-based analysis of 10 character elements (e.g., worldview) to structure internal cognition; (2) Reference-Guided Reward, which utilizes overlap-based metrics with reference responses as optimization anchors to enhance exploration and performance; and (3) Character-Conditioned Reward Normalization, which adjusts reward distributions based on character categories to ensure robust optimization across heterogeneous roles. Extensive experiments demonstrate that Character-R1 significantly outperforms existing methods in knowledge, memory and others.