mlsanityData to 7 Oct 2026

VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models

arXiv 2503.07575 ↗First posted 10 Mar 2025cs.CV cs.CLcited by 14

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

This research investigates both explicit and implicit social biases exhibited by Vision-Language Models (VLMs). The key distinction between these bias types lies in the level of awareness: explicit bias refers to conscious, intentional biases, while implicit bias operates subconsciously. To analyze explicit bias, we directly pose questions to VLMs related to gender and racial differences: (1) Multiple-choice questions based on a given image (e.g., "What is the education level of the person in the image?") (2) Yes-No comparisons using two images (e.g., "Is the person in the first image more educated than the person in the second image?") For implicit bias, we design tasks where VLMs assist users but reveal biases through their responses: (1) Image description tasks: Models are asked to describe individuals in images, and we analyze disparities in textual cues across demographic groups. (2) Form completion tasks: Models draft a personal information collection form with 20 attributes, and we examine correlations among selected attributes for potential biases. We evaluate Gemini-1.5, GPT-4V, GPT-4o, LLaMA-3.2-Vision and LLaVA-v1.6. Our code and data are publicly available at https://github.com/uscnlp-lime/VisBias.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 57 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

13 of 57 references.

References · 13

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2023Foundational Models in Medical Imaging: A Comprehensive Survey and Future Visioncited by 74
  7. 2024MultiMath: Bridging Visual and Mathematical Reasoning for Large Language Modelscited by 69
  8. 2025Ocean-OCR: Towards General OCR Application via a Vision-Language Modelcited by 24
  9. 2024MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Modelcited by 20
  10. 2023Multimodal Foundation Models Exploit Text to Make Medical Image Predictionscited by 19
  11. 2024BIGbench: A Unified Benchmark for Evaluating Multi-dimensional Social Biases in Text-to-Image Modelscited by 17
  12. 2024Fairness Testing of Large Language Models in Role-Playingcited by 10
1 more
  1. 2025FairCoder: Probing LLM Bias in High-Stakes Decision Making via Coding Taskscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 9.

14 papers.

Sort:

By area:

  1. 2026The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Modelscited by 6
  2. 2025DSO: Direct Steering Optimization for Bias Mitigationcited by 3
  3. 2025VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Modelscited by 2
  4. 2025From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationshipcited by 2
  5. 2025Bias Beyond Demographics: Probing Decision Boundaries in Black-Box LVLMs via Counterfactual VQAcited by 2
  6. 2025Large Language Models Develop Novel Social Biases Through Adaptive Explorationcited by 2
  7. 2026Enhancing Lightweight Vision Language Models through Group Competitive Learning for Socially Compliant Navigationcited by 0
  8. 2026EgoGroups: A Benchmark For Detecting Social Groups of People in the Wildcited by 0
  9. 2026Demographic and Linguistic Bias Evaluation in Omnimodal Language Modelscited by 0
  10. 2026Identifying Ethical Biases in Action Recognition Modelscited by 0
  11. 2026Artificial Intolerance: Stigmatizing Language in Clinical Documentation Skews Large Language Model Decision-Makingcited by 0
  12. 2026Mitigating Bias in Large Vision-Language Models via Counterfactual Ensemble Decodingcited by 0
2 more
  1. 2026Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Modelscited by 0
  2. 2026FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Makingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024BiasDora: Exploring Hidden Biased Associations in Vision-Language Models
  2. 2024Identifying Implicit Social Biases in Vision-Language Models
  3. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals
  4. 2023MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Models
  5. 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models
  6. 2024Uncovering Bias in Large Vision-Language Models with Counterfactuals
  7. 2025Ask Me Again Differently: GRAS for Measuring Bias in Vision Language Models on Gender, Race, Age, and Skin Tone
  8. 2024Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective