mlsanityData to 14 Oct 2026

A Stereotype Content Analysis on Color-related Social Bias in Large Vision Language Models

arXiv 2505.20901 ↗First posted 27 May 2025cs.CL cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

As large vision language models(LVLMs) rapidly advance, concerns about their potential to learn and generate social biases and stereotypes are increasing. Previous studies on LVLM's stereotypes face two primary limitations: metrics that overlooked the importance of content words, and datasets that overlooked the effect of color. To address these limitations, this study introduces new evaluation metrics based on the Stereotype Content Model (SCM). We also propose BASIC, a benchmark for assessing gender, race, and color stereotypes. Using SCM metrics and BASIC, we conduct a study with eight LVLMs to discover stereotypes. As a result, we found three findings. (1) The SCM-based evaluation is effective in capturing stereotypes. (2) LVLMs exhibit color stereotypes in the output along with gender and race ones. (3) Interaction between model architecture and parameter sizes seems to affect stereotypes. We release BASIC publicly on [anonymized for review].

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 52 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

24 of 52 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts25 Jun
  2. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals30 May
  3. 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models21 Feb
  4. 2024Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images8 Feb
  5. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples30 Nov
  6. 2023Debiasing Vision-Language Models via Biased Prompts31 Jan

Other references · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  6. 2023Social Biases through the Text-to-Image Generation Lenscited by 127
  7. 2023Generative Language Models Exhibit Social Identity Biasescited by 111
  8. 2024VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignmentcited by 58
  9. 2024Understanding the Impact of Negative Prompts: When and How Do They Take Effect?cited by 51
  10. 2024Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgerycited by 39
  11. 2025ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustnesscited by 28
  12. 2023Gender Biases in Automatic Evaluation Metrics for Image Captioningcited by 24
6 more
  1. 2021Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Modelcited by 23
  2. 2024textttModSCAN: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalitiescited by 22
  3. 2024New Job, New Gender? Measuring the Social Bias in Image Generation Modelscited by 16
  4. 2025Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Imagescited by 14
  5. 2020Color Channel Perturbation Attacks for Fooling Convolutional Neural Networks and A Defense Against Such Attackscited by 10
  6. 2025When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contextscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Cultural Counterfactuals: Evaluating Cultural Biases in Large Vision-Language Models with Counterfactual Examplescited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Visual Cues of Gender and Race are Associated with Stereotyping in Vision-Language Models
  2. 2024More Distinctively Black and Feminine Faces Lead to Increased Stereotyping in Vision-Language Models
  3. 2023Controlling for Stereotypes in Multimodal Language Model Evaluation
  4. 2023MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Models
  5. 2025VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models
  6. 2025VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models
  7. 2024VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model
  8. 2025Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models