# textttModSCAN: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalities

arXiv: https://arxiv.org/abs/2410.06967 · HTML: https://mlsanity.com/paper/2410.06967/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2024.

## Built on by (most cited first, 22 of 22)

- [Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency](https://mlsanity.com/paper/2510.21189/) (2025)
- [BBQ-V: Benchmarking Visual Stereotype Bias in Large Multimodal Models](https://mlsanity.com/paper/2502.08779/) (2025)
- [AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment](https://mlsanity.com/paper/2509.11620/) (2025)
- [Sparse Models, Sparse Safety: Unsafe Routes in Mixture-of-Experts LLMs](https://mlsanity.com/paper/2602.08621/) (2026)
- [Real Money, Fake Models: Deceptive Model Claims in Shadow APIs](https://mlsanity.com/paper/2603.01919/) (2026)
- [Interpretable Debiasing of Vision-Language Models for Social Fairness](https://mlsanity.com/paper/2602.24014/) (2026)
- [Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation](https://mlsanity.com/paper/2509.07596/) (2025)
- [Characterizing Selective Refusal Bias in Large Language Models](https://mlsanity.com/paper/2510.27087/) (2025)
- [VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models](https://mlsanity.com/paper/2505.22897/) (2025)
- [Moral Sycophancy in Vision Language Models](https://mlsanity.com/paper/2602.08311/) (2026)
- [VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech](https://mlsanity.com/paper/2604.17248/) (2026)
- [Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics](https://mlsanity.com/paper/2601.04946/) (2026)
- [When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm](https://mlsanity.com/paper/2603.24079/) (2026)
- [Data Matters Most: Auditing Social Bias in Contrastive Vision Language Models](https://mlsanity.com/paper/2501.13223/) (2025)
- [A Stereotype Content Analysis on Color-related Social Bias in Large Vision Language Models](https://mlsanity.com/paper/2505.20901/) (2025)
- [Interpreting Social Bias in LVLMs via Information Flow Analysis and Multi-Round Dialogue Evaluation](https://mlsanity.com/paper/2505.21106/) (2025)
- [One-shot Optimized Steering Vector for Hallucination Mitigation for VLMs](https://mlsanity.com/paper/2601.23041/) (2026)
- [StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs](https://mlsanity.com/paper/2606.20527/) (2026)
- [Is GPT-4o mini Blinded by its Own Safety Filters? Exposing the Multimodal-to-Unimodal Bottleneck in Hate Speech Detection](https://mlsanity.com/paper/2509.13608/) (2025)
- [The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training](https://mlsanity.com/paper/2604.07754/) (2026)
- [Mitigating Bias in Large Vision-Language Models via Counterfactual Ensemble Decoding](https://mlsanity.com/paper/2608.21415/) (2026)
- [Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models](https://mlsanity.com/paper/2608.29590/) (2026)

## Abstract (arXiv)

Large vision-language models (LVLMs) have been rapidly developed and widely used in various fields, but the (potential) stereotypical bias in the model is largely unexplored. In this study, we present a pioneering measurement framework, textttModSCAN, to underlineSCAN the stereotypical bias within LVLMs from both vision and language underlineModalities. textttModSCAN examines stereotypical biases with respect to two typical stereotypical attributes (gender and race) across three kinds of scenarios: occupations, descriptors, and persona traits. Our findings suggest that 1) the currently popular LVLMs show significant stereotype biases, with CogVLM emerging as the most biased model; 2) these stereotypical biases may stem from the inherent biases in the training dataset and pre-trained models; 3) the utilization of specific prompt prefixes (from both vision and language modalities) performs well in reducing stereotypical biases. We believe our work can serve as the foundation for understanding and addressing stereotypical bias in LVLMs.