mlsanityData to 14 Oct 2026

Revealing Persona Biases in Dialogue Systems

arXiv 2104.08728 ↗First posted 18 Apr 2021cs.CLcited by 33

Not on the map. Why?

Abstract (arXiv)

Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people's lives. Modern dialogue systems may consider adopting anthropomorphic personas, mimicking societal demographic groups to appear more approachable and trustworthy to users. However, the adoption of a persona can result in the adoption of biases. In this paper, we present the first large-scale study on persona biases in dialogue systems and conduct analyses on personas of different social classes, sexual orientations, races, and genders. We define persona biases as harmful differences in responses (e.g., varying levels of offensiveness, agreement with harmful statements) generated from adopting different demographic personas. Furthermore, we introduce an open-source framework, UnitPersonaBias, to explore and aggregate persona biases in dialogue systems. By analyzing the Blender and DialoGPT dialogue systems, we observe that adopting personas can actually decrease harmful responses, compared to not using any personas. Additionally, we find that persona choices can affect the degree of harms in generated responses and thus should be systematically evaluated before deployment. We also analyze how personas can result in different amounts of harm towards specific demographics.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 22 of its 35 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

22 of 35 references.

References · 22

  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  2. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  4. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  5. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  6. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  7. 2020Recipes for building an open-domain chatbotcited by 593
  8. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  9. 2018Gender Bias in Coreference Resolutioncited by 442
  10. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  11. 2016A Persona-Based Neural Conversation Modelcited by 223
  12. 2018Gender Bias in Neural Natural Language Processingcited by 211
10 more
  1. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  2. 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
  3. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  4. 2017ParlAI: A Dialog Research Software Platformcited by 139
  5. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  6. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  7. 2018Conversational AI: The Science Behind the Alexa Prizecited by 63
  8. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
  9. 2017Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Modelscited by 12
  10. 2018Towards Automated Customer Supportcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 11; 2024: 6; 2025: 5; 2026: 1.

33 papers.

Sort:

By area:

  1. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  2. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  3. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  4. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  5. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
  6. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
  7. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 78
  8. 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 53
  9. 2024Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attributioncited by 40
  10. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  11. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  12. 2024Identifying and Improving Disability Bias in GPT-Based Resume Screeningcited by 28
21 more
  1. 2023Survey of Social Bias in Vision-Language Modelscited by 26
  2. 2021Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue modelscited by 23
  3. 2023CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Modelscited by 23
  4. 2024CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Modelscited by 19
  5. 2024IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answeringcited by 19
  6. 2021Alexa, Google, Siri: What are Your Pronouns? Gender and Anthropomorphism in the Design and Perception of Conversational Assistantscited by 16
  7. 2023Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Modelcited by 14
  8. 2024Ask LLMs Directly, "What shapes your bias?": Measuring Social Bias in Large Language Modelscited by 14
  9. 2021The Arabic Parallel Gender Corpus 2.0: Extensions and Analysescited by 12
  10. 2025DAIQ: Auditing Demographic Attribute Inference from Question in LLMscited by 10
  11. 2022User-Centric Gender Rewritingcited by 7
  12. 2022Towards Robust NLG Bias Evaluation with Syntactically-diverse Promptscited by 7
  13. 2023Exposing Bias in Online Communities through Large-Scale Language Modelscited by 5
  14. 2025Localizing Persona Representations in LLMscited by 3
  15. 2025Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detectioncited by 2
  16. 2023Reducing Sensitivity on Speaker Names for Text Generation from Dialoguescited by 1
  17. 2023Healing Unsafe Dialogue Responses with Weak Supervision Signalscited by 1
  18. 2024Refusal as Silence: Gendered Disparities in Vision-Language Model Responsescited by 1
  19. 2025Exploring the Impact of Occupational Personas on Domain-Specific QAcited by 1
  20. 2025"As Eastern Powers, I will veto." : An Investigation of Nation-level Bias of Large Language Models in International Relationscited by 1
  21. 2026Playing Games with My Heart: An Evaluation of AI Companion Appscited by 0