How Robust Is Homogeneity Bias in LLMs? Evidence Across Models, Decoding Settings, and Identity Signals
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) reproduce homogeneity bias -- the tendency to portray marginalized groups as more internally similar than dominant groups -- but whether this bias generalizes across models, is stable under different inference settings, or depends on how group identity is signaled remains unstudied. We map homogeneity bias across seven open-weight instruction-tuned LLMs (7-20B parameters), a 5x5 temperature x top-p decoding grid, and two paradigms for signaling group identity (explicit labels vs. racially distinctive names). In six of seven models, Hispanic and Asian Americans are portrayed as significantly more homogeneous than White Americans at the default configuration, and the effect remains positive on average at every temperature and top-p tested; African American and gender bias instead vary in direction across models. A conservative cell-level re-analysis confirms Hispanic and Asian homogeneity as robust while weaker African American and gender signals largely do not survive, establishing group-specific robustness. We also apply the same grid to a names-based paradigm in which group identity is signaled via racially distinctive surnames rather than explicit labels. The names paradigm corroborates Hispanic and Asian homogeneity bias, but Black-coded surnames elicit robustly less homogeneous outputs than White-coded names in every model tested -- a reversal absent from the label paradigm -- showing that how group identity is operationalized shapes which biases surface and in which direction.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 47 references that are in the map. First posted on arXiv 4 Jan 2025.
Builds on
12 of 47 references.
References · 12
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
- 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
- 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 97
- 2024What's in a Name? Auditing Large Language Models for Race and Gender Biascited by 51
- 2024Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humanscited by 31
- 2024Vision-Language Models Generate More Homogeneous Stories for Phenotypically Black Individualscited by 6
Built on by
Not cited yet.