# Persona is a Double-edged Sword: Mitigating the Negative Impact of Role-playing Prompts in Zero-shot Reasoning Tasks

arXiv: https://arxiv.org/abs/2408.08631 · HTML: https://mlsanity.com/paper/2408.08631/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 16 Aug 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 8 of 8)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [Split and Merge: Aligning Position Biases in LLM-based Evaluators](https://mlsanity.com/paper/2310.01432/) (2023)
- [When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models](https://mlsanity.com/paper/2311.10054/) (2023)

## Built on by (most cited first, 28 of 28)

- [Voting or Consensus? Decision-Making in Multi-Agent Debate](https://mlsanity.com/paper/2502.19130/) (2025)
- [LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios](https://mlsanity.com/paper/2508.17692/) (2025)
- [Creativity in LLM-based Multi-Agent Systems: A Survey](https://mlsanity.com/paper/2505.21116/) (2025)
- [How RLHF Amplifies Sycophancy](https://mlsanity.com/paper/2602.01002/) (2026)
- [Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents](https://mlsanity.com/paper/2510.04491/) (2025)
- [Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning](https://mlsanity.com/paper/2511.06190/) (2025)
- [HARBOR: Exploring Persona Dynamics in Multi-Agent Competition](https://mlsanity.com/paper/2502.12149/) (2025)
- [German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies](https://mlsanity.com/paper/2511.21722/) (2025)
- [BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Models](https://mlsanity.com/paper/2408.10608/) (2024)
- [Do Persona-Infused LLMs Affect Performance in a Strategic Reasoning Game?](https://mlsanity.com/paper/2512.06867/) (2025)
- [Can LLMs Reliably Simulate Real Students' Abilities in Mathematics and Reading Comprehension?](https://mlsanity.com/paper/2507.08232/) (2025)
- [Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI](https://mlsanity.com/paper/2511.00230/) (2025)
- [The Persona Paradox: Medical Personas as Behavioral Priors in Clinical Language Models](https://mlsanity.com/paper/2601.05376/) (2026)
- [Persona Prompting as a Lens on LLM Social Reasoning](https://mlsanity.com/paper/2601.20757/) (2026)
- [Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?](https://mlsanity.com/paper/2511.21218/) (2025)
- [A Role-Aware Multi-Agent Framework for Financial Education Question Answering with LLMs](https://mlsanity.com/paper/2509.09727/) (2025)
- [CHOIR: Collaborative Harmonization fOr Inference Robustness](https://mlsanity.com/paper/2510.22475/) (2025)
- [Persona-based Multi-Agent Collaboration for Brainstorming](https://mlsanity.com/paper/2512.04488/) (2025)
- [PerMix-RLVR: Preserving Persona Expressivity under Verifiable-Reward Alignment](https://mlsanity.com/paper/2604.08986/) (2026)
- [Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models](https://mlsanity.com/paper/2604.13206/) (2026)
- [When Does Critique Improve AI-Assisted Theoretical Physics? SCALAR: Structured Critic--Actor Loop for Agentic Reasoning](https://mlsanity.com/paper/2605.06772/) (2026)
- [The Virtual Roundtable: Multi-Agent Personas Simulating the Dynamics of Human Brainstorming](https://mlsanity.com/paper/2606.05178/) (2026)
- [Decision Protocols in Multi-Agent Large Language Model Conversations](https://mlsanity.com/paper/2607.05477/) (2026)
- [L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning](https://mlsanity.com/paper/2607.09099/) (2026)
- [Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents](https://mlsanity.com/paper/2607.15095/) (2026)
- [The Analyst in the Prompt: Role, Retrieval, and Memory Biases in LLM Financial Analysis](https://mlsanity.com/paper/2609.03218/) (2026)
- [Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models](https://mlsanity.com/paper/2609.39853/) (2026)
- [OPSRD: On-Policy Self-Role Distillation](https://mlsanity.com/paper/2609.39884/) (2026)

## Abstract (arXiv)

Recent studies demonstrate that prompting a role-playing persona to an LLM improves reasoning capability. However, assigning an adequate persona is difficult since LLMs are extremely sensitive to assigned prompts; thus, inaccurately defined personas sometimes hinder LLMs and degrade their reasoning capabilities. In this paper, we first investigate the potential negative impact of injecting persona into language models. Furthermore, we propose a novel framework, Jekyll & Hyde, which ensembles the outcomes of both role-playing and neutral prompts to enhance the robustness of reasoning ability. Specifically, Jekyll & Hyde predicts an appropriate persona using an LLM when defining the role-playing prompt. Then, Jekyll & Hyde collects two potential solutions from role-playing and neutral prompts and selects a better solution using the LLM evaluator. The experimental analysis demonstrates that role-playing prompts sometimes distract LLMs, degrading their reasoning abilities in 7 out of 12 datasets in llama3. Meanwhile, Jekyll & Hyde improve reasoning capabilities by selecting better choices among the potential solutions on twelve widely-used natural language reasoning datasets. In addition, we reveal that assigning LLM-generated personas obtains more stable results than handcrafted personas.