# Revealing Persona Biases in Dialogue Systems

arXiv: https://arxiv.org/abs/2104.08728 · HTML: https://mlsanity.com/paper/2104.08728/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 18 Apr 2021.

## Builds on (most cited first, 22 of 22)

- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [A Persona-Based Neural Conversation Model](https://mlsanity.com/paper/1603.06155/) (2016)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack](https://mlsanity.com/paper/1908.06083/) (2019)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation](https://mlsanity.com/paper/1911.03842/) (2019)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [Does Gender Matter? Towards Fairness in Dialogue Systems](https://mlsanity.com/paper/1910.10486/) (2019)
- [Ethical Challenges in Data-Driven Dialogue Systems](https://mlsanity.com/paper/1711.09050/) (2017)
- [Conversational AI: The Science Behind the Alexa Prize](https://mlsanity.com/paper/1801.03604/) (2018)
- [Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning](https://mlsanity.com/paper/2009.13028/) (2020)
- [Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models](https://mlsanity.com/paper/1710.07388/) (2017)
- [Towards Automated Customer Support](https://mlsanity.com/paper/1809.00303/) (2018)

## Built on by (most cited first, 30 of 33)

- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization](https://mlsanity.com/paper/2212.10465/) (2022)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling](https://mlsanity.com/paper/2107.03451/) (2021)
- [BiasAsker: Measuring the Bias in Conversational AI System](https://mlsanity.com/paper/2305.12434/) (2023)
- [Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution](https://mlsanity.com/paper/2403.03121/) (2024)
- [Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems](https://mlsanity.com/paper/2310.05280/) (2023)
- [Evaluation of African American Language Bias in Natural Language Generation](https://mlsanity.com/paper/2305.14291/) (2023)
- [Identifying and Improving Disability Bias in GPT-Based Resume Screening](https://mlsanity.com/paper/2402.01732/) (2024)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models](https://mlsanity.com/paper/2109.03300/) (2021)
- [CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models](https://mlsanity.com/paper/2306.16244/) (2023)
- [CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models](https://mlsanity.com/paper/2402.15265/) (2024)
- [IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answering](https://mlsanity.com/paper/2408.13545/) (2024)
- [Alexa, Google, Siri: What are Your Pronouns? Gender and Anthropomorphism in the Design and Perception of Conversational Assistants](https://mlsanity.com/paper/2106.02578/) (2021)
- [Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model](https://mlsanity.com/paper/2305.11140/) (2023)
- [Ask LLMs Directly, "What shapes your bias?": Measuring Social Bias in Large Language Models](https://mlsanity.com/paper/2406.04064/) (2024)
- [The Arabic Parallel Gender Corpus 2.0: Extensions and Analyses](https://mlsanity.com/paper/2110.09216/) (2021)
- [DAIQ: Auditing Demographic Attribute Inference from Question in LLMs](https://mlsanity.com/paper/2508.15830/) (2025)
- [User-Centric Gender Rewriting](https://mlsanity.com/paper/2205.02211/) (2022)
- [Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts](https://mlsanity.com/paper/2212.01700/) (2022)
- [Exposing Bias in Online Communities through Large-Scale Language Models](https://mlsanity.com/paper/2306.02294/) (2023)
- [Localizing Persona Representations in LLMs](https://mlsanity.com/paper/2505.24539/) (2025)
- [Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection](https://mlsanity.com/paper/2502.06487/) (2025)
- [Reducing Sensitivity on Speaker Names for Text Generation from Dialogues](https://mlsanity.com/paper/2305.13833/) (2023)
- [Healing Unsafe Dialogue Responses with Weak Supervision Signals](https://mlsanity.com/paper/2305.15757/) (2023)
- [Refusal as Silence: Gendered Disparities in Vision-Language Model Responses](https://mlsanity.com/paper/2406.08222/) (2024)

## Abstract (arXiv)

Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people's lives. Modern dialogue systems may consider adopting anthropomorphic personas, mimicking societal demographic groups to appear more approachable and trustworthy to users. However, the adoption of a persona can result in the adoption of biases. In this paper, we present the first large-scale study on persona biases in dialogue systems and conduct analyses on personas of different social classes, sexual orientations, races, and genders. We define persona biases as harmful differences in responses (e.g., varying levels of offensiveness, agreement with harmful statements) generated from adopting different demographic personas. Furthermore, we introduce an open-source framework, UnitPersonaBias, to explore and aggregate persona biases in dialogue systems. By analyzing the Blender and DialoGPT dialogue systems, we observe that adopting personas can actually decrease harmful responses, compared to not using any personas. Additionally, we find that persona choices can affect the degree of harms in generated responses and thus should be systematically evaluated before deployment. We also analyze how personas can result in different amounts of harm towards specific demographics.