mlsanityData to 7 Oct 2026

StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs

arXiv 2606.20527 ↗First posted 18 Jun 2026cs.CL cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

Multimodal large language models (MLLMs) are increasingly deployed in personally and societally consequential settings, yet the visual cues that shape how these models judge people remain poorly understood. Prior work often compares different (groups of) individuals, making it difficult to separate appearance effects from identity differences. We introduce StylisticBias, a controlled benchmark for evaluating attribute-level social bias in MLLMs. We generate 500 photorealistic base faces and create about 50 single-attribute variations per face, producing about 25K images. This design keeps identity fixed and changes one visual attribute at a time. It lets us measure how specific cues shift model judgments. We evaluate six MLLMs across 25 binary social judgment scenarios. We find that age and body type dominate identity-level effects, while fashion style and other visual cues drive the largest attribute-level shifts. We further find that about 15 attributes account for nearly 80% of the total variation, showing that bias is concentrated in a small set of visual cues. Sensitivity is strongest in judgments that are semantically aligned with appearance, especially socioeconomic and style-related judgments. We release StylisticBias as a benchmark for fine-grained bias evaluation in multimodal models. Code and dataset: https://github.com/timo-cavelius/StylisticBias and https://hf.co/datasets/shaghayegh/stylistic-bias-dataset.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 54 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

22 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos11 Jan
  2. 2025Bias Beyond Demographics: Probing Decision Boundaries in Black-Box LVLMs via Counterfactual VQA5 Aug
  3. 2023VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution21 Jun

Other references · 19

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  3. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  4. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  5. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  6. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  7. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  8. 2024OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelscited by 61
  9. 2023TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Modelscited by 51
  10. 2024textttModSCAN: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalitiescited by 21
  11. 2025AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessmentcited by 10
  12. 2025Beauty and the Bias: Exploring the Impact of Attractiveness on Multimodal Large Language Modelscited by 7
7 more
  1. 2025MR. Judge: Multimodal Reasoner as a Judgecited by 7
  2. 2025CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMscited by 5
  3. 2025LLMs Reproduce Stereotypes of Sexual and Gender Minoritiescited by 4
  4. 2025Framing the Game: How Context Shapes LLM Decision-Makingcited by 4
  5. 2025Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifierscited by 4
  6. 2026Quantifying and Mitigating Socially Desirable Responding in LLMs: A Desirability-Matched Graded Forced-Choice Psychometric Studycited by 2
  7. 2026Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ContextBias: Controlled Evaluation of Bias Persistence Under Context Shift in Text-to-Image Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models
  2. 2024Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI
  3. 2024Social Perception of Faces in a Vision-Language Model
  4. 2026Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness
  5. 2025Visual Cues of Gender and Race are Associated with Stereotyping in Vision-Language Models
  6. 2025Bias in the Picture: Benchmarking VLMs with Social-Cue News Images and LLM-as-Judge Assessment
  7. 2025VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models
  8. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals