mlsanityData to 14 Oct 2026

Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models

arXiv 2505.14160 ↗First posted 20 May 2025cs.CL cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

Multilingual vision-language models (VLMs) promise universal image-text retrieval, yet their social biases remain underexplored. We perform the first systematic audit of four public multilingual CLIP variants: M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and the debiased SigLIP-2, covering ten languages that differ in resource availability and morphological gender marking. Using balanced subsets of FairFace and the PATA stereotype suite in a zero-shot setting, we quantify race and gender bias and measure stereotype amplification. Contrary to the intuition that multilinguality mitigates bias, every model exhibits stronger gender skew than its English-only baseline. CAPIVARA-CLIP shows its largest biases precisely in the low-resource languages it targets, while the shared encoder of NLLB-CLIP and SigLIP-2 transfers English gender stereotypes into gender-neutral languages; loosely coupled encoders largely avoid this leakage. Although SigLIP-2 reduces agency and communion skews, it inherits -- and in caption-sparse contexts (e.g., Xhosa) amplifies -- the English anchor's crime associations. Highly gendered languages consistently magnify all bias types, yet gender-neutral languages remain vulnerable whenever cross-lingual weight sharing imports foreign stereotypes. Aggregated metrics thus mask language-specific hot spots, underscoring the need for fine-grained, language-aware bias evaluation in future multilingual VLM research.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 35 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

18 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Identifying Implicit Social Biases in Vision-Language Models1 Nov
  2. 2024FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models28 Sep
  3. 2023DeAR: Debiasing Vision-Language Models with Additive Residuals18 Mar
  4. 2023Debiasing Vision-Language Models via Biased Prompts31 Jan

Other references · 14

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  3. 2023Large Language Modelscited by 780
  4. 2019Evaluating Gender Bias in Machine Translationcited by 249
  5. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  6. 2024FairCLIP: Harnessing Fairness in Vision-Language Learningcited by 69
  7. 2023Comparing Biases and the Impact of Multilingual Training across Multiple Languagescited by 36
  8. 2024CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?cited by 35
  9. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 35
  10. 2024FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSscited by 25
  11. 2023Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scalecited by 24
  12. 2023NLLB-CLIP -- train performant multilingual image retrieval model on a budgetcited by 24
2 more
  1. 2024Do Multilingual Large Language Models Mitigate Stereotype Bias?cited by 13
  2. 2024Mitigating Social Bias in Large Language Models: A Multi-Objective Approach within a Multi-Agent Frameworkcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Investigating Social Bias in Narrative Image Generationcited by 1
  2. 2026GLEaN: A Text-to-image Bias Detection Approach for Public Comprehensioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Data Matters Most: Auditing Social Bias in Contrastive Vision Language Models
  2. 2026Gender Bias in Vision-Language In-Context Learning
  3. 2023Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities
  4. 2023MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Models
  5. 2024Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images
  6. 2023VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution
  7. 2025Freeze and Reveal: Exposing Modality Bias in Vision-Language Models
  8. 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models