mlsanityData to 14 Oct 2026

I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models

arXiv 2402.10436 ↗First posted 16 Feb 2024cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

We explored cultural biases-individualism vs. collectivism-in ChatGPT across three Western languages (i.e., English, German, and French) and three Eastern languages (i.e., Chinese, Japanese, and Korean). When ChatGPT adopted an individualistic persona in Western languages, its collectivism scores (i.e., out-group values) exhibited a more negative trend, surpassing their positive orientation towards individualism (i.e., in-group values). Conversely, when a collectivistic persona was assigned to ChatGPT in Eastern languages, a similar pattern emerged with more negative responses toward individualism (i.e., out-group values) as compared to collectivism (i.e., in-group values). The results indicate that when imbued with a particular social identity, ChatGPT discerns in-group and out-group, embracing in-group values while eschewing out-group values. Notably, the negativity towards the out-group, from which prejudices and discrimination arise, exceeded the positivity towards the in-group. The experiment was replicated in the political domain, and the results remained consistent. Furthermore, this replication unveiled an intrinsic Democratic bias in Large Language Models (LLMs), aligning with earlier findings and providing integral insights into mitigating such bias through prompt engineering. Extensive robustness checks were performed using varying hyperparameter and persona setup methods, with or without social identity labels, across other popular language models.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 42 of its 111 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

42 of 111 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  8. 2019How Can We Know What Language Models Know?cited by 959
  9. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  10. 2023Whose Opinions Do Language Models Reflect?cited by 773
  11. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  12. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
30 more
  1. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 260
  2. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 233
  3. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 199
  4. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
  5. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  6. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 148
  7. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 148
  8. 2022Large Language Models Meet NL2Code: A Surveycited by 138
  9. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  10. 2023Factuality Challenges in the Era of Large Language Modelscited by 126
  11. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
  12. 2023Generative Language Models Exhibit Social Identity Biasescited by 111
  13. 2022Human heuristics for AI-generated language are flawedcited by 105
  14. 2014The Rise of Social Botscited by 96
  15. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
  16. 2023AI model GPT-3 (dis)informs us better than humanscited by 78
  17. 2017The spread of low-credibility content by social botscited by 73
  18. 2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
  19. 2022In conversation with Artificial Intelligence: aligning language models with human valuescited by 62
  20. 2023Multilingual Language Models are not Multicultural: A Case Study in Emotioncited by 54
  21. 2023Towards Interpretable Mental Health Analysis with Large Language Modelscited by 53
  22. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  23. 2022Cultural Incongruencies in Artificial Intelligencecited by 21
  24. 2017Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Mediacited by 18
  25. 2023''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Textcited by 15
  26. 2023Shaping the Emerging Norms of Using Large Language Models in Social Computing Researchcited by 14
  27. 2022The Myth of Culturally Agnostic AI Modelscited by 11
  28. 2023ChatGPT Is More Likely to Be Perceived as Male Than Femalecited by 5
  29. 2023ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Managementcited by 3
  30. 2023Red AI? Inconsistent Responses from GPT3.5 Models on Political Issues in the US and Chinacited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Plurals: A System for Guiding LLMs Via Simulated Social Ensemblescited by 17
  2. 2024The high dimensional psychological profile and cultural bias of ChatGPTcited by 2
  3. 2026Toward a social psychology of AI: language-model agents reproduce human-like minimal-group biascited by 2
  4. 2026LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systemscited by 1
  5. 2025A Multifaceted Analysis of Social Biases in Large Language Modelscited by 0