# Generative Language Models Exhibit Social Identity Biases

arXiv: https://arxiv.org/abs/2310.15819 · HTML: https://mlsanity.com/paper/2310.15819/

## Key facts

- Cited by 111 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 113 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 30 of 44)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [UL2: Unifying Language Learning Paradigms](https://mlsanity.com/paper/2205.05131/) (2022)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)

## Built on by (most cited first, 30 of 111)

- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [Can LLM be a Personalized Judge?](https://mlsanity.com/paper/2406.11657/) (2024)
- [Systematic Biases in LLM Simulations of Debates](https://mlsanity.com/paper/2402.04049/) (2024)
- [Do LLM Agents Exhibit Social Behavior?](https://mlsanity.com/paper/2312.15198/) (2023)
- [SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors](https://mlsanity.com/paper/2510.17516/) (2025)
- [PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs](https://mlsanity.com/paper/2503.09543/) (2025)
- [Conformity in Large Language Models](https://mlsanity.com/paper/2410.12428/) (2024)
- [Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets](https://mlsanity.com/paper/2410.07991/) (2024)
- [A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians](https://mlsanity.com/paper/2405.20956/) (2024)
- [An Evaluation of Cultural Value Alignment in LLM](https://mlsanity.com/paper/2504.08863/) (2025)
- [The simulation of judgment in LLMs](https://mlsanity.com/paper/2502.04426/) (2025)
- [Group size effects and collective misalignment in LLM multi-agent systems](https://mlsanity.com/paper/2510.22422/) (2025)
- [C3AI: Crafting and Evaluating Constitutions for Constitutional AI](https://mlsanity.com/paper/2502.15861/) (2025)
- [Evaluating LLMs in Finance Requires Explicit Bias Consideration](https://mlsanity.com/paper/2602.14233/) (2026)
- [ValuesRAG: Enhancing Cultural Alignment Through Retrieval-Augmented Contextual Learning](https://mlsanity.com/paper/2501.01031/) (2025)
- [Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging](https://mlsanity.com/paper/2510.17426/) (2025)
- [AI use in American newspapers is widespread, uneven, and rarely disclosed](https://mlsanity.com/paper/2510.18774/) (2025)
- [Silenced Biases: The Dark Side LLMs Learned to Refuse](https://mlsanity.com/paper/2511.03369/) (2025)
- [Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoption](https://mlsanity.com/paper/2409.03843/) (2024)
- [FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes](https://mlsanity.com/paper/2506.23111/) (2025)
- [Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity](https://mlsanity.com/paper/2507.00657/) (2025)
- [Language Models Predict Empathy Gaps Between Social In-groups and Out-groups](https://mlsanity.com/paper/2503.01030/) (2025)
- [Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays](https://mlsanity.com/paper/2503.20062/) (2025)
- [A Review of Incorporating Psychological Theories in LLMs](https://mlsanity.com/paper/2505.00003/) (2025)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models](https://mlsanity.com/paper/2402.10436/) (2024)
- [GPT-ology, Computational Models, Silicon Sampling: How should we think about LLMs in Cognitive Science?](https://mlsanity.com/paper/2406.09464/) (2024)
- [How far can bias go? Tracing bias from pretraining data to alignment](https://mlsanity.com/paper/2411.19240/) (2024)
- [Systematic Bias in Large Language Models: Discrepant Response Patterns in Binary vs. Continuous Judgment Tasks](https://mlsanity.com/paper/2504.19445/) (2025)

## Abstract (arXiv)

The surge in popularity of large language models has given rise to concerns about biases that these models could learn from humans. We investigate whether ingroup solidarity and outgroup hostility, fundamental social identity biases known from social psychology, are present in 56 large language models. We find that almost all foundational language models and some instruction fine-tuned models exhibit clear ingroup-positive and outgroup-negative associations when prompted to complete sentences (e.g., "We are..."). Our findings suggest that modern language models exhibit fundamental social identity biases to a similar degree as humans, both in the lab and in real-world conversations with LLMs, and that curating training data and instruction fine-tuning can mitigate such biases. Our results have practical implications for creating less biased large-language models and further underscore the need for more research into user interactions with LLMs to prevent potential bias reinforcement in humans.