mlsanityData to 7 Oct 2026

Better Zero-Shot Reasoning with Role-Play Prompting

arXiv 2308.07702 ↗First posted 15 Aug 2023cs.CLcited by 220

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Modern large language models (LLMs) exhibit a remarkable capacity for role-playing, enabling them to embody not only human characters but also non-human entities. This versatility allows them to simulate complex human-like interactions and behaviors within various contexts, as well as to emulate specific objects or systems. While these capabilities have enhanced user engagement and introduced novel modes of interaction, the influence of role-playing on LLMs' reasoning abilities remains underexplored. In this study, we introduce a strategically designed role-play prompting methodology and assess its performance under the zero-shot setting across twelve diverse reasoning benchmarks. Our empirical results illustrate that role-play prompting consistently surpasses the standard zero-shot approach across most datasets. Notably, in experiments conducted using ChatGPT, accuracy on AQuA rises from 53.5% to 63.8%, and on Last Letter from 23.8% to 84.2%.Upon further comparison with the Zero-Shot-CoT technique, which prompts the model to "think step by step", our study demonstrates that role-play prompting acts as a more effective trigger for the CoT process. This highlights its potential to augment the reasoning capabilities of LLMs. We release our code at https://github.com/NKU-HLT/Role-Play-Prompting.

Key facts. Cited by 220 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 26 of its 32 references that are in the map. First posted on arXiv 15 Aug 2023.

Builds on

26 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
  2. 2022Making Large Language Models Better Reasoners with Step-Aware Verifier6 Jun

Other references · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
12 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  4. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  5. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  6. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  7. 2016Solving General Arithmetic Word Problemscited by 353
  8. 2023Role-Play with Large Language Modelscited by 290
  9. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  10. 2023Large Language Models are Diverse Role-Players for Summarization Evaluationcited by 62
  11. 2023Better Zero-Shot Reasoning with Self-Adaptive Promptingcited by 52
  12. 2022Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterancescited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 59; 2025: 92; 2026: 60.

220 papers.

Sort:

By area:

  1. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  2. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
  3. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  4. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 89
  5. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  6. 2024How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environmentscited by 66
  7. 2023Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Valuescited by 49
  8. 2024Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Modelscited by 44
  9. 2024SUGARCREPE++ Dataset: Vision-Language Model Sensitivity to Semantic and Lexical Alterationscited by 42
  10. 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Educationcited by 38
  11. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  12. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attentioncited by 32
38 more
  1. 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
  2. 2024Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Biascited by 28
  3. 2024Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agentscited by 26
  4. 2023LLMs as Hackers: Autonomous Linux Privilege Escalation Attackscited by 25
  5. 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
  6. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agentscited by 22
  7. 2024TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Modelscited by 21
  8. 2024Helpful assistant or fruitful facilitator? Investigating how personas affect language model behaviorcited by 21
  9. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learningcited by 21
  10. 2023Evaluating ChatGPT as a Recommender System: A Rigorous Approachcited by 20
  11. 2024I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBenchcited by 20
  12. 2025Population-Aligned Persona Generation for LLM-based Social Simulationcited by 20
  13. 2024How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHOcited by 19
  14. 2024Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Modelscited by 19
  15. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learningcited by 19
  16. 2025On the Effect of Sampling Diversity in Scaling LLM Inferencecited by 18
  17. 2024The Sound of Healthcare: Improving Medical Transcription ASR Accuracy with Large Language Modelscited by 17
  18. 2024Quantifying the Capabilities of LLMs across Scale and Precisioncited by 17
  19. 2024PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Drivingcited by 17
  20. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 17
  21. 2025Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performancecited by 17
  22. 2025ViDDAR: Vision Language Model-Based Task-Detrimental Content Detection for Augmented Realitycited by 16
  23. 2025CARE: Multilingual Human Preference Learning for Cultural Awarenesscited by 15
  24. 2024Ask LLMs Directly, "What shapes your bias?": Measuring Social Bias in Large Language Modelscited by 14
  25. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinkingcited by 14
  26. 2025Large Language Models are Autonomous Cyber Defenderscited by 14
  27. 2025To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Promptcited by 14
  28. 2024Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Modelscited by 13
  29. 2024Multimodal Pragmatic Jailbreak on Text-to-image Modelscited by 13
  30. 2024Self-Prompt Tuning: Enable Autonomous Role-Playing in LLMscited by 12
  31. 2025Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoningcited by 12
  32. 2025Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactionscited by 12
  33. 2024Toward Adaptive Reasoning in Large Language Models with Thought Rollbackcited by 11
  34. 2025Improving LLM Reasoning through Interpretable Role-Playing Steeringcited by 11
  35. 2025K2-Think: A Parameter-Efficient Reasoning Systemcited by 11
  36. 2024Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Modelscited by 10
  37. 2024CharacterGPT: A Persona Reconstruction Framework for Role-Playing Agentscited by 10
  38. 2025Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Datacited by 10

50 most cited of 220 All 220 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning Does Not Necessarily Improve Role-Playing Ability
  2. 2026Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models
  3. 2024Persona is a Double-edged Sword: Mitigating the Negative Impact of Role-playing Prompts in Zero-shot Reasoning Tasks
  4. 2025RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following
  5. 2024PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models
  6. 2024Thinking Before Speaking: A Role-playing Model with Mindset
  7. 2024Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment
  8. 2025RAGs to Riches: RAG-like Few-shot Learning for Large Language Model Role-playing