# Fairness Testing of Large Language Models in Role-Playing

arXiv: https://arxiv.org/abs/2411.00585 · HTML: https://mlsanity.com/paper/2411.00585/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 28 of 28)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024)
- [Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles](https://mlsanity.com/paper/2407.00870/) (2024)
- [LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play](https://mlsanity.com/paper/2405.06373/) (2024)
- [Fairness Testing: A Comprehensive Survey and Analysis of Trends](https://mlsanity.com/paper/2207.10223/) (2022)
- [Fairness in Large Language Models: A Taxonomic Survey](https://mlsanity.com/paper/2404.01349/) (2024)
- [BiasAsker: Measuring the Bias in Conversational AI System](https://mlsanity.com/paper/2305.12434/) (2023)
- [Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness](https://mlsanity.com/paper/2005.12379/) (2020)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models](https://mlsanity.com/paper/2310.12481/) (2023)
- [NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification](https://mlsanity.com/paper/2112.13214/) (2021)
- [A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifiers](https://mlsanity.com/paper/2207.03277/) (2022)
- [Fairness-aware Configuration of Machine Learning Libraries](https://mlsanity.com/paper/2202.06196/) (2022)
- [BiasFinder: Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systems](https://mlsanity.com/paper/2102.01859/) (2021)
- [Fair Enough: Searching for Sufficient Measures of Fairness](https://mlsanity.com/paper/2110.13029/) (2021)
- [Can LLMs Correct Physicians, Yet? Investigating Effective Interaction Methods in the Medical Domain](https://mlsanity.com/paper/2403.20288/) (2024)
- ["A Woman is More Culturally Knowledgeable than A Man?": The Effect of Personas on Cultural Norm Interpretation in LLMs](https://mlsanity.com/paper/2409.11636/) (2024)

## Built on by (most cited first, 11 of 11)

- [Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.00306/) (2025)
- [VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models](https://mlsanity.com/paper/2503.07575/) (2025)
- [Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue History](https://mlsanity.com/paper/2505.21362/) (2025)
- [Can LLMs Reason Structurally? Benchmarking via the Lens of Data Structures](https://mlsanity.com/paper/2505.24069/) (2025)
- [BiasCause: Evaluate Socially Biased Causal Reasoning of Large Language Models](https://mlsanity.com/paper/2504.07997/) (2025)
- [Relative Bias: A Comparative Framework for Quantifying Bias in LLMs](https://mlsanity.com/paper/2505.17131/) (2025)
- [PapersPlease: A Benchmark for Evaluating Motivational Values of Large Language Models Based on ERG Theory](https://mlsanity.com/paper/2506.21961/) (2025)
- ["As Eastern Powers, I will veto." : An Investigation of Nation-level Bias of Large Language Models in International Relations](https://mlsanity.com/paper/2511.10695/) (2025)
- [Identifying and Mitigating Gender Cues in Academic Recommendation Letters: An Interpretability Case Study](https://mlsanity.com/paper/2604.12337/) (2026)
- [StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs](https://mlsanity.com/paper/2605.10442/) (2026)
- [Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation](https://mlsanity.com/paper/2609.05993/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have become foundational in modern language-driven software applications, profoundly influencing daily life. A critical technique in leveraging their potential is role-playing, where LLMs simulate diverse roles to enhance their real-world utility. However, while research has highlighted the presence of social biases in LLM outputs, it remains unclear whether and to what extent these biases emerge during role-playing scenarios. In this paper, we conduct an empirical study on fairness testing of LLMs in role-playing scenarios. To enable this testing, we use LLMs to generate 550 social roles spanning a comprehensive set of 11 demographic attributes, producing 33,000 role-specific questions that target various forms of bias. These questions, covering Yes/No, multiple-choice, and open-ended formats, are designed to prompt LLMs to adopt specific roles and respond accordingly. We employ a combination of rule-based and LLM-based strategies to identify biased responses, rigorously validated through human evaluation. Using the generated questions as the test cases, we conduct extensive evaluations of 10 advanced LLMs. The evaluation reveal 107,580 biased responses across the studied LLMs, with individual models yielding between 7,579 and 16,963 biased responses, underscoring the prevalence of bias in role-playing contexts. To support future research, we have publicly released the dataset, along with all scripts and experimental results.