mlsanityData to 7 Oct 2026

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models

arXiv 2310.00746 ↗First posted 1 Oct 2023cs.CL cs.AIcited by 173

Not on the map. Why?

Cited by 46 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters. However, the closed-source nature of state-of-the-art LLMs and their general-purpose training limit role-playing optimization. In this paper, we introduce RoleLLM, a framework to benchmark, elicit, and enhance role-playing abilities in LLMs. RoleLLM comprises four stages: (1) Role Profile Construction for 100 roles; (2) Context-Based Instruction Generation (Context-Instruct) for role-specific knowledge extraction; (3) Role Prompting using GPT (RoleGPT) for speaking style imitation; and (4) Role-Conditioned Instruction Tuning (RoCIT) for fine-tuning open-source models along with role customization. By Context-Instruct and RoleGPT, we create RoleBench, the first systematic and fine-grained character-level benchmark dataset for role-playing with 168,093 samples. Moreover, RoCIT on RoleBench yields RoleLLaMA (English) and RoleGLM (Chinese), significantly enhancing role-playing abilities and even achieving comparable results with RoleGPT (using GPT-4).

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). Builds on 41 of its 65 references that are in the map. First posted on arXiv 1 Oct 2023.

Builds on

41 of 65 references.

References · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
29 more
  1. 2023LIMA: Less Is More for Alignmentcited by 1,080
  2. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  3. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  4. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  5. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  6. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  7. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  8. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  9. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  10. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  11. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  12. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  13. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  14. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  15. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  16. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  17. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  18. 2023GPTScore: Evaluate as You Desirecited by 357
  19. 2023Tool Learning with Foundation Modelscited by 302
  20. 2023Role-Play with Large Language Modelscited by 290
  21. 2022Large Language Models with Controllable Working Memorycited by 169
  22. 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 161
  23. 2023Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Casescited by 85
  24. 2021SummScreen: A Dataset for Abstractive Screenplay Summarizationcited by 80
  25. 2023ChatHaruhi: Reviving Anime Character in Reality via Large Language Modelcited by 57
  26. 2023Multi-Party Chat: Conversational Agents in Group Settings with Humans and Modelscited by 53
  27. 2023Chinese Open Instruction Generalist: A Preliminary Releasecited by 35
  28. 2023Interactive Natural Language Processingcited by 31
  29. 2023ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Humancited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 56; 2025: 51; 2026: 62.

173 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  3. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  4. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 142
  5. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  6. 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 121
  7. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  8. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  9. 2024PowerInfer-2: Fast Large Language Model Inference on a Smartphonecited by 101
  10. 2024Hymba: A Hybrid-head Architecture for Small Language Modelscited by 95
  11. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  12. 2024MoVA: Adapting Mixture of Vision Experts to Multimodal Contextcited by 82
38 more
  1. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  2. 2025LLM Generated Persona is a Promise with a Catchcited by 77
  3. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  4. 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Seriescited by 67
  5. 2024LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Playcited by 65
  6. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  7. 2024HelloBench: Evaluating Long Text Generation Capabilities of Large Language Modelscited by 42
  8. 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systemscited by 38
  9. 2024Personality Alignment of Large Language Modelscited by 36
  10. 2024ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agentscited by 35
  11. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 32
  12. 2024AI PERSONA: Towards Life-long Personalization of LLMscited by 30
  13. 2024Emotional RAG: Enhancing Role-Playing Agents through Emotional Retrievalcited by 26
  14. 2025CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulationcited by 26
  15. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  16. 2024DDK: Distilling Domain Knowledge for Efficient Large Language Modelscited by 25
  17. 2025OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personascited by 25
  18. 2023LARP: Language-Agent Role Play for Open-World Gamescited by 24
  19. 2024MIO: A Foundation Model on Multimodal Tokenscited by 24
  20. 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
  21. 2024Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrappingcited by 22
  22. 2024Helpful assistant or fruitful facilitator? Investigating how personas affect language model behaviorcited by 21
  23. 2024MMRole: A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agentscited by 21
  24. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  25. 2025Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Gamescited by 20
  26. 2024Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playingcited by 17
  27. 2024Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models with Personality-Indicative Datacited by 17
  28. 2024A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Educationcited by 16
  29. 2026Learning Personalized Agents from Human Feedbackcited by 16
  30. 2024ESC-Eval: Evaluating Emotion Support Conversations in Large Language Modelscited by 15
  31. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agentscited by 15
  32. 2024LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultationcited by 14
  33. 2024Can Large Language Models Grasp Legal Theories? Enhance Legal Reasoning with Insights from Multi-Agent Collaborationcited by 14
  34. 2025Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoningcited by 14
  35. 2024Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Plannercited by 12
  36. 2024PersoBench: Benchmarking Personalized Response Generation in Large Language Modelscited by 12
  37. 2025E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Modelscited by 12
  38. 2024Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspectivecited by 11

50 most cited of 173 All 173 (JSON)