mlsanityData to 14 Oct 2026

Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoption

arXiv 2409.03843 ↗First posted 5 Sep 2024cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Drawing parallels between human cognition and artificial intelligence, we explored how large language models (LLMs) internalize identities imposed by targeted prompts. Informed by Social Identity Theory, these identity assignments lead LLMs to distinguish between "we" (the ingroup) and "they" (the outgroup). This self-categorization generates both ingroup favoritism and outgroup bias. Nonetheless, existing literature has predominantly focused on ingroup favoritism, often overlooking outgroup bias, which is a fundamental source of intergroup prejudice and discrimination. Our experiment addresses this gap by demonstrating that outgroup bias manifests as strongly as ingroup favoritism. Furthermore, we successfully mitigated the inherent pro-liberal, anti-conservative bias in LLMs by guiding them to adopt the perspectives of the initially disfavored group. These results were replicated in the context of gender bias. Our findings highlight the potential to develop more equitable and balanced language models.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 31 of its 75 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

31 of 75 references.

References · 31

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  5. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  6. 2023Whose Opinions Do Language Models Reflect?cited by 773
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  8. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  9. 2022Using cognitive psychology to understand GPT-3cited by 398
  10. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 260
  11. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 199
  12. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 189
19 more
  1. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
  2. 2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 172
  3. 2023Factuality Challenges in the Era of Large Language Modelscited by 126
  4. 2023Generative Language Models Exhibit Social Identity Biasescited by 111
  5. 2024Cognitive Bias in Decision-Making with LLMscited by 106
  6. 2022Human heuristics for AI-generated language are flawedcited by 105
  7. 2023Deception Abilities Emerged in Large Language Modelscited by 101
  8. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 97
  9. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  10. 2022Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identitycited by 74
  11. 2017The spread of low-credibility content by social botscited by 73
  12. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  13. 2023ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Modelscited by 54
  14. 2024Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Modelscited by 45
  15. 2023Bias of AI-Generated Content: An Examination of News Produced by Large Language Modelscited by 38
  16. 2022CommunityLM: Probing Partisan Worldviews from Language Modelscited by 36
  17. 2023''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Textcited by 15
  18. 2024Temporal Blind Spots in Large Language Modelscited by 7
  19. 2022Beyond Digital "Echo Chambers": The Role of Viewpoint Diversity in Political Discussioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 3.

9 papers.

  1. 2025FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypescited by 9
  2. 2025Language Models Predict Empathy Gaps Between Social In-groups and Out-groupscited by 6
  3. 2025A Review of Incorporating Psychological Theories in LLMscited by 6
  4. 2025Modeling Subjectivity in Cognitive Appraisal with Language Modelscited by 3
  5. 2025Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM-Generated Textcited by 2
  6. 2026Truth or Tribe: How In-group Favoritism Prioritize Facts in Persona Agentscited by 2
  7. 2026Toward a social psychology of AI: language-model agents reproduce human-like minimal-group biascited by 2
  8. 2025The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Modelscited by 0
  9. 2026Why LLM Agents Favor Their Group: Stakes, Observed Norms, and Reputationcited by 0