mlsanityData to 14 Oct 2026

KuaiRP Series Role-playing Models Technical Report

arXiv 2609.11127 ↗First posted 10 Sep 2026cs.AI cs.CLcited by 0

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

This paper introduces the complete technical solution for the KuaiRP series of role-playing models. We aim to achieve four core objectives for a dedicated role-playing model: simplified prompt engineering, highly stable output quality, built-in domain world knowledge, and high-efficiency deployment with a small parameter size. However, effectively injecting deep domain knowledge often leads to a severe catastrophic forgetting of the model's general agent capabilities. To overcome this trade-off, we propose a multi-stage training pipeline. First, we design a standardized character template and construct an SFT data pipeline based on user behavior simulation and reverse profile filtering. Next, we utilize a rule-based composite reward function during the Reinforcement Learning (RL) phase to eliminate common degradation phenomena like length expansion and repetitive generation. Finally, to recover the general capabilities compromised during SFT and RL, we propose a novel self-distillation paradigm using Two-stage On-Policy Distillation (OPD) equipped with Cumulative-Divergence Decay (CDD). By using the domain-adapted model as the teacher and the original base model as the student, we effectively balance deep domain knowledge injection with the preservation of general agent capabilities. Experimental results demonstrate that the KuaiRP models not only match the current state-of-the-art proprietary models in role-playing fidelity within our target domains, but also successfully recover general agent capabilities, maintaining extremely low deployment costs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 10 of its 14 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

10 of 14 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026On the Position Bias of On-Policy Distillation21 Jun
  2. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation1 Jun
  3. 2026Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance29 May
  4. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models26 Jan

Other references · 6

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  6. 2026TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
  2. 2026X-KD: General Experiential Knowledge Distillation for Large Language Models
  3. 2026Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
  4. 2026Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
  5. 2024Role Play: Learning Adaptive Role-Specific Strategies in Multi-Agent Interactions
  6. 2026Adversarial Closed-Loop Curriculum for Evolving Role-Playing Agents
  7. 2026CADENT: Gated Hybrid Distillation for Sample-Efficient Transfer in Reinforcement Learning
  8. 2026HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing