# Vision-Language Models Generate More Homogeneous Stories for Phenotypically Black Individuals

arXiv: https://arxiv.org/abs/2412.09668 · HTML: https://mlsanity.com/paper/2412.09668/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Social bias and fairness in VLMs](https://mlsanity.com/mllm/social-bias-and-fairness-in-vlms/)

## Builds on (most cited first, 13 of 13)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale](https://mlsanity.com/paper/2211.03759/) (2022)
- [Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies](https://mlsanity.com/paper/2108.12084/) (2021)
- [Social Biases through the Text-to-Image Generation Lens](https://mlsanity.com/paper/2304.06034/) (2023)
- [xGen-MM (BLIP-3): A Family of Open Large Multimodal Models](https://mlsanity.com/paper/2408.08872/) (2024)
- [Understanding and Evaluating Racial Biases in Image Captioning](https://mlsanity.com/paper/2106.08503/) (2021)
- [CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations](https://mlsanity.com/paper/2310.11501/) (2023)
- [On Measures of Biases and Harms in NLP](https://mlsanity.com/paper/2108.03362/) (2021)
- [Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans](https://mlsanity.com/paper/2401.08495/) (2024)
- [Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasets](https://mlsanity.com/paper/1910.13268/) (2019)
- [Smiling Women Pitching Down: Auditing Representational and Presentational Gender Biases in Image Generative AI](https://mlsanity.com/paper/2305.10566/) (2023)

## Built on by (most cited first, 6 of 6)

- [Bias Amplification in Stable Diffusion's Representation of Stigma Through Skin Tones and Their Homogeneity](https://mlsanity.com/paper/2508.17465/) (2025)
- [VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models](https://mlsanity.com/paper/2505.22897/) (2025)
- [How Robust Is Homogeneity Bias in LLMs? Evidence Across Models, Decoding Settings, and Identity Signals](https://mlsanity.com/paper/2501.02211/) (2025)
- [Token-Level Entropy Reveals Demographic Disparities in Large Language Models](https://mlsanity.com/paper/2501.19337/) (2025)
- [Breaking the Assistant Mold: Modeling Behavioral Variation in LLM Based Procedural Character Generation](https://mlsanity.com/paper/2601.03396/) (2026)
- [PERSONAWEAVER: Controllable Diversity Beyond Conventional Archetypes in Procedural Character Generation](https://mlsanity.com/paper/2609.26629/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) extend Large Language Models' capabilities by integrating image processing, but concerns persist about their potential to reproduce and amplify human biases. While research has documented how these models perpetuate stereotypes across demographic groups, most work has focused on between-group biases rather than within-group differences. This study investigates homogeneity bias-the tendency to portray groups as more uniform than they are-within Black Americans, examining how perceived racial phenotypicality influences VLMs' outputs. Using computer-generated images that systematically vary in phenotypicality, we prompted VLMs to generate stories about these individuals and measured text similarity to assess content homogeneity. Our findings reveal three key patterns: First, VLMs generate significantly more homogeneous stories about Black individuals with higher phenotypicality compared to those with lower phenotypicality. Second, stories about Black women consistently display greater homogeneity than those about Black men across all models tested. Third, in two of three VLMs, this homogeneity bias is primarily driven by a pronounced interaction where phenotypicality strongly influences content variation for Black women but has minimal impact for Black men. These results demonstrate how intersectionality shapes AI-generated representations and highlight the persistence of stereotyping that mirror documented biases in human perception, where increased racial phenotypicality leads to greater stereotyping and less individualized representation.