A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas
Not on the map. Why?
Abstract (arXiv)
As LLMs (large language models) are increasingly used to generate synthetic personas particularly in data-limited domains such as health, privacy, and HCI, it becomes necessary to understand how these narratives represent identity, especially that of minority communities. In this paper, we audit synthetic personas generated by 3 LLMs (GPT4o, Gemini 1.5 Pro, Deepseek 2.5) through the lens of representational harm, focusing specifically on racial identity. Using a mixed methods approach combining close reading, lexical analysis, and a parameterized creativity framework, we compare 1512 LLM generated personas to human-authored responses. Our findings reveal that LLMs disproportionately foreground racial markers, overproduce culturally coded language, and construct personas that are syntactically elaborate yet narratively reductive. These patterns result in a range of sociotechnical harms, including stereotyping, exoticism, erasure, and benevolent bias, that are often obfuscated by superficially positive narrations. We formalize this phenomenon as algorithmic othering, where minoritized identities are rendered hypervisible but less authentic. Based on these findings, we offer design recommendations for narrative-aware evaluation metrics and community-centered validation protocols for synthetic identity generation.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 20 of its 72 references that are in the map. First posted on arXiv 7 May 2025.
Builds on
20 of 72 references.
References · 20
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
- 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 249
- 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
- 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
- 2021On Measures of Biases and Harms in NLPcited by 81
- 2024A Complete Survey on LLM-based AI Chatbotscited by 71
- 2023AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asiacited by 54
- 2024A Synthetic Dataset for Personal Attribute Inferencecited by 36
- 2024Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humanscited by 31
- 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 28
- 2024Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agentscited by 26
8 more
- 2024Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approachcited by 26
- 2021A Sociotechnical View of Algorithmic Fairnesscited by 24
- 2024ChatGPT as Research Scientist: Probing GPT's Capabilities as a Research Librarian, Research Ethicist, Data Generator and Data Predictorcited by 21
- 2024From Melting Pots to Misrepresentations: Exploring Harms in Generative AIcited by 13
- 2023The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysiscited by 12
- 2025Thousand Voices of Trauma: A Large-Scale Synthetic Dataset for Modeling Prolonged Exposure Therapy Conversationscited by 11
- 2025The Impostor is Among Us: Can Large Language Models Capture the Complexity of Human Personas?cited by 5
- 2024Concerns on Bias in Large Language Models when Creating Synthetic Personaecited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 7.
8 papers.
- 2026The Need for a Socially-Grounded Persona Framework for User Simulationcited by 15
- 2026One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalizationcited by 9
- 2026Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contextscited by 6
- 2025Measuring Lexical Diversity of Synthetic Data Generated through Fine-Grained Persona Promptingcited by 4
- 2026What if AI systems weren't chatbots?cited by 2
- 2026PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AIcited by 0
- 2026On Defining Erasure Harms for NLPcited by 0
- 2026Who Should Be Generated? Justifying Demographic Targets in Open-Ended Generationcited by 0
- 2026Who Should Be Generated? Justifying Demographic Targets in Open-Ended Generation3 Aug 2026
- 2026On Defining Erasure Harms for NLP14 Jun 2026
- 2026What if AI systems weren't chatbots?8 May 2026
- 2026PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI7 May 2026
- 2026Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contexts3 Feb 2026
- 2026One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization26 Jan 2026
- 2026The Need for a Socially-Grounded Persona Framework for User Simulation12 Jan 2026
- 2025Measuring Lexical Diversity of Synthetic Data Generated through Fine-Grained Persona Prompting23 May 2025