# Role-Play Paradox in Large Language Models: Reasoning Performance Gains and Ethical Dilemmas

arXiv: https://arxiv.org/abs/2409.13979 · HTML: https://mlsanity.com/paper/2409.13979/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 21 Sep 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation](https://mlsanity.com/paper/2310.17389/) (2023)
- [On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning](https://mlsanity.com/paper/2212.08061/) (2022)
- [Gender Bias in Large Language Models across Multiple Languages](https://mlsanity.com/paper/2403.00277/) (2024)
- [Identification of Bias Against People with Disabilities in Sentiment Analysis and Toxicity Detection Models](https://mlsanity.com/paper/2111.13259/) (2021)
- [Self-Prompt Tuning: Enable Autonomous Role-Playing in LLMs](https://mlsanity.com/paper/2407.08995/) (2024)
- [White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs](https://mlsanity.com/paper/2404.10508/) (2024)
- [Modeling the Sacred: Considerations when Using Religious Texts in Natural Language Processing](https://mlsanity.com/paper/2404.14740/) (2024)

## Built on by (most cited first, 6 of 6)

- [CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs](https://mlsanity.com/paper/2505.11413/) (2025)
- [Reasoning Towards Fairness: Mitigating Bias in Language Models through Reasoning-Guided Fine-Tuning](https://mlsanity.com/paper/2504.05632/) (2025)
- [PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality](https://mlsanity.com/paper/2606.20897/) (2026)
- [Enhancing Exchange Rate Forecasting with Explainable Deep Learning Models](https://mlsanity.com/paper/2410.19241/) (2024)
- [Harm in AI-Driven Societies: An Audit of Toxicity Adoption on Chirper.ai](https://mlsanity.com/paper/2601.01090/) (2026)
- [Role-SynthCLIP: A Role Play Driven Diverse Synthetic Data Approach](https://mlsanity.com/paper/2511.05057/) (2025)

## Abstract (arXiv)

Role-play in large language models (LLMs) enhances their ability to generate contextually relevant and high-quality responses by simulating diverse cognitive perspectives. However, our study identifies significant risks associated with this technique. First, we demonstrate that autotuning, a method used to auto-select models' roles based on the question, can lead to the generation of harmful outputs, even when the model is tasked with adopting neutral roles. Second, we investigate how different roles affect the likelihood of generating biased or harmful content. Through testing on benchmarks containing stereotypical and harmful questions, we find that role-play consistently amplifies the risk of biased outputs. Our results underscore the need for careful consideration of both role simulation and tuning processes when deploying LLMs in sensitive or high-stakes contexts.