mlsanityData to 14 Oct 2026

Disparities In Negation Understanding Across Languages In Vision-Language Models

arXiv 2604.18942 ↗First posted 21 Apr 2026cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) exhibit affirmation bias: a systematic tendency to select positive captions ("X is present") even when the correct description contains negation ("no X"). While prior work has documented this failure mode in English and proposed solutions, negation manifests differently across languages through varying morphology, word order, and cliticization patterns, raising the question of whether these solutions serve all linguistic communities equitably. We introduce the first human-verified multilingual negation benchmark, spanning seven typologically diverse languages: English, Mandarin Chinese, Arabic, Greek, Russian, Tagalog, and Spanish. Evaluating three VLMs - CLIP, SigLIP, and MultiCLIP - we find that standard CLIP performs at or below chance on non-Latin-script languages, while MultiCLIP achieves the highest and most uniform accuracy. We also evaluate SpaceVLM, a proposed negation correction, and find that it produces substantial improvements for several languages - particularly English, Greek, Spanish, and Tagalog - while showing varied effectiveness across typologically different languages. This variation reveals that linguistic properties like morphology, script, and negation structure interact with model improvements in fairness-relevant ways. As VLMs are deployed globally, multilingual benchmarks are essential for understanding not just whether solutions work, but for whom.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 10 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

8 of 10 references.

References · 8

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
  5. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 120
  6. 2025Vision-Language Models Do Not Understand Negationcited by 72
  7. 2023Towards Bridging the Digital Language Dividecited by 7
  8. 2025SpaceVLM: Sub-Space Modeling of Negation in Vision-Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Learn "No" to Say "Yes" Better: Improving Vision-Language Models via Negations
  2. 2026When Negation Is a Geometry Problem in Vision-Language Models
  3. 2025A Multimodal Recaptioning Framework to Account for Perceptual Diversity Across Languages in Vision-Language Modeling
  4. 2025Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models
  5. 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understanding
  6. 2024Why do LLaVA Vision-Language Models Reply to Images in English?
  7. 2025NegVQA: Can Vision Language Models Understand Negation?
  8. 2026Language-Conditioned Visual Grounding with CLIP Multilingual