mlsanityData to 14 Oct 2026

Same evidence, different judgments: Evidence noncommutative in vision/speech-text conflicts

arXiv 2609.26986 ↗First posted 22 Sep 2026cs.AIcited by 0

New in Multimodal LLMs / vision-language models; no direction yet. Multimodal LLMs / vision-language models

Abstract (arXiv)

For multimodal large language models, when images or speech conflict with accompanying text, measured text reliance can entangle modality preference with evidence position. Earlier studies of text bias often used a fixed evidence order or moved task instructions with the evidence, leaving the contribution of order unclear. In this paper, we use a paired comparison that keeps the instructions and evidence content fixed and swaps only the positions of the two sources to quantify this potential influence. Across vision and speech models, placing an image or recording after conflicting text consistently shifts answers toward its content. We also revisit previous studies and analyze why their experimental settings can lead to misleading conclusions. These findings reveal cross-modal evidence noncommutativity: the same evidence can lead to different judgments when its order changes, and placing perceptual evidence later can increase the model's reliance on its content.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 35 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

31 of 35 references.

References · 31

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  6. 2025Qwen2.5-Omni Technical Reportcited by 736
  7. 2024Qwen2-Audio Technical Reportcited by 560
  8. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  9. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
  10. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  11. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 193
  12. 2024Pixtral 12Bcited by 179
19 more
  1. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 89
  2. 2022Can Large Language Models Truly Understand Prompts? A Case Study with Negated Promptscited by 59
  3. 2024Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMscited by 38
  4. 2024Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Modelscited by 28
  5. 2024Order Matters: Exploring Order Sensitivity in Multimodal Large Language Modelscited by 20
  6. 2025Robust Multimodal Large Language Models Against Modality Conflictcited by 18
  7. 2025When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Modelscited by 13
  8. 2024Correcting Negative Bias in Large Language Models through Negative Attention Score Alignmentcited by 12
  9. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflictcited by 10
  10. 2026Do Images Speak Louder than Words? Investigating the Effect of Textual Misinformation in VLMscited by 7
  11. 2025Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Modelscited by 5
  12. 2026Jailbreaking Vision-Language Models Through the Visual Modalitycited by 4
  13. 2026PathMark: Protecting Intellectual Property of Mixture-of-Expert LLMs via Path Watermarkscited by 4
  14. 2026Do System Prompts Leave Behavioral Fingerprints? A Large-Scale Empirical Study of Clone Detection via Output Similaritycited by 4
  15. 2026FragileFlow: Spectral Control of Correct-but-Fragile Predictions for Foundation Model Robustnesscited by 3
  16. 2025Rethinking Multi-Agent Intelligence Through the Lens of Small-World Networkscited by 2
  17. 2026Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Promptscited by 2
  18. 2026SCOPE: Sequential Conformal Probing for Reliable OOD Rejection in LLM Servicescited by 2
  19. 2026Same Semantics, Different Outcome: On the Modality Robustness of Multimodal LLMs under Knowledge Conflictcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers
  2. 2026When Audio-LLMs Don't Listen: A Cross-Linguistic Study of Modality Arbitration
  3. 2026Which Source Wins? Task-Dependent Reliance in Vision-Language Models
  4. 2026Real Images, Worse Judgments: Evaluating Vision-Language Models on Concreteness and Imagery
  5. 2026When Words Speak Louder than Images: Towards Understanding Language Bias in Vision-Language Models
  6. 2026The Evidence Anchoring Boundary: Where Visual Interventions Matter and When They Help in Multimodal LLMs
  7. 2026Visual Evidence Under Cross-Examination: Evaluating and Controlling Decision-Level Evidence Use in Vision-Language Models
  8. 2026Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts