mlsanityData to 14 Oct 2026

Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers

arXiv 2610.19407 ↗First posted 14 Oct 2026cs.CL cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs also fits: VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models must reconcile conflicting text and images. Prior work has characterized the resulting behavior in terms of modality biases--e.g., a "blind faith in text"--but why preferences vary across models, domains, and input conditions remains unclear. We develop a predictive account from a Bayesian perspective, positing that VLMs combine conditionally independent evidence from each modality with prior beliefs, subject to modality-specific biases. The resulting two-parameter cue combination model accurately predicts VLM preferences under unseen combinations of input degradation and transfers to new forms of unreliability without refitting. Separating evidence strength from integration bias explains how VLMs can follow text even while weighting visual evidence more heavily, reconciling reports of text and vision dominance under a single combination rule.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 46 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

16 of 46 references.

References · 16

  1. 2015VQA: Visual Question Answeringcited by 2,506
  2. 2019Language Models as Knowledge Bases?cited by 1,936
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  5. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 92
  6. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 89
  7. 2024What's in the Image? A Deep-Dive into the Vision of Vision Language Modelscited by 68
  8. 2024Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMscited by 38
  9. 2025Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfactscited by 32
  10. 2025How Do Vision-Language Models Process Conflicting Information Across Modalities?cited by 23
  11. 2025When Modalities Conflict: How Unimodal Reasoning Uncertainty Governs Preference Dynamics in MLLMscited by 21
  12. 2017DeepCorrect: Correcting DNN models against Image Distortionscited by 15
4 more
  1. 2026Arbitration Failure, Not Perceptual Blindness: How Vision-Language Models Resolve Visual-Linguistic Conflictscited by 11
  2. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflictcited by 10
  3. 2025MaLoRA: Gated Modality LoRA for Key-Space Alignment in Multimodal LLM Fine-Tuningcited by 4
  4. 2026Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Same evidence, different judgments: Evidence noncommutative in vision/speech-text conflicts
  2. 2026When Words Speak Louder than Images: Towards Understanding Language Bias in Vision-Language Models
  3. 2026Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
  4. 2026V-FAT: Benchmarking Visual Fidelity Against Text-bias
  5. 2026Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts
  6. 2026Which Source Wins? Task-Dependent Reliance in Vision-Language Models
  7. 2026Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models
  8. 2026When Vision-Language Models Judge Without Seeing: Exposing Informativeness Bias