# A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models

arXiv: https://arxiv.org/abs/2402.13636 · HTML: https://mlsanity.com/paper/2402.13636/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Social bias and fairness in VLMs](https://mlsanity.com/mllm/social-bias-and-fairness-in-vlms/)

## Builds on (most cited first, 30 of 32)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [Measuring and Reducing Gendered Correlations in Pre-trained Models](https://mlsanity.com/paper/2010.06032/) (2020)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [Any-to-Any Generation via Composable Diffusion](https://mlsanity.com/paper/2305.11846/) (2023)
- [A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle](https://mlsanity.com/paper/1901.10002/) (2019)
- [Towards Debiasing Sentence Representations](https://mlsanity.com/paper/2007.08100/) (2020)
- [ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts](https://mlsanity.com/paper/2312.00784/) (2023)
- [No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World](https://mlsanity.com/paper/1711.08536/) (2017)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges](https://mlsanity.com/paper/2311.03287/) (2023)
- [Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases](https://mlsanity.com/paper/2010.15052/) (2020)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images](https://mlsanity.com/paper/2402.05779/) (2024)
- [Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2104.08666/) (2021)
- [VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution](https://mlsanity.com/paper/2306.12424/) (2023)
- [Mitigating Gender Bias in Captioning Systems](https://mlsanity.com/paper/2006.08315/) (2020)
- [Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models](https://mlsanity.com/paper/1912.00578/) (2019)
- ['Person' == Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion](https://mlsanity.com/paper/2310.19981/) (2023)
- [Parameter-efficient Modularised Bias Mitigation via AdapterFusion](https://mlsanity.com/paper/2302.06321/) (2023)

## Built on by (most cited first, 11 of 11)

- [VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model](https://mlsanity.com/paper/2406.14194/) (2024)
- [Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations](https://mlsanity.com/paper/2410.13976/) (2024)
- [Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation](https://mlsanity.com/paper/2509.07596/) (2025)
- [DSO: Direct Steering Optimization for Bias Mitigation](https://mlsanity.com/paper/2512.15926/) (2025)
- [Bias Beyond Demographics: Probing Decision Boundaries in Black-Box LVLMs via Counterfactual VQA](https://mlsanity.com/paper/2508.03079/) (2025)
- [debiaSAE: Benchmarking and Mitigating Vision-Language Model Bias](https://mlsanity.com/paper/2410.13146/) (2024)
- [A Stereotype Content Analysis on Color-related Social Bias in Large Vision Language Models](https://mlsanity.com/paper/2505.20901/) (2025)
- [Surgeons Are Indian Males and Speech Therapists Are White Females: Auditing Biases in Vision-Language Models for Healthcare Professionals](https://mlsanity.com/paper/2510.06280/) (2025)
- [Debias your Large Multi-Modal Model at Test-Time via Non-Contrastive Visual Attribute Steering](https://mlsanity.com/paper/2411.12590/) (2024)
- [Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models](https://mlsanity.com/paper/2608.29590/) (2026)
- [LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark](https://mlsanity.com/paper/2609.00192/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs) have gained widespread adoption in both industry and academia. In this study, we propose a unified framework for systematically evaluating gender, race, and age biases in VLMs with respect to professions. Our evaluation encompasses all supported inference modes of the recent VLMs, including image-to-text, text-to-text, text-to-image, and image-to-image. Additionally, we propose an automated pipeline to generate high-quality synthetic datasets that intentionally conceal gender, race, and age information across different professional domains, both in generated text and images. The dataset includes action-based descriptions of each profession and serves as a benchmark for evaluating societal biases in vision-language models (VLMs). In our comparative analysis of widely used VLMs, we have identified that varying input-output modalities lead to discernible differences in bias magnitudes and directions. Additionally, we find that VLM models exhibit distinct biases across different bias attributes we investigated. We hope our work will help guide future progress in improving VLMs to learn socially unbiased representations. We will release our data and code.