# Same evidence, different judgments: Evidence noncommutative in vision/speech-text conflicts

arXiv: https://arxiv.org/abs/2609.26986 · HTML: https://mlsanity.com/paper/2609.26986/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 35 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models

## Builds on (most cited first, 30 of 31)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [SLURP: A Spoken Language Understanding Resource Package](https://mlsanity.com/paper/2011.13205/) (2020)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [Pixtral 12B](https://mlsanity.com/paper/2410.07073/) (2024)
- [Words or Vision: Do Vision-Language Models Have Blind Faith in Text?](https://mlsanity.com/paper/2503.02199/) (2025)
- [Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts](https://mlsanity.com/paper/2209.12711/) (2022)
- [Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs](https://mlsanity.com/paper/2410.08145/) (2024)
- [Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models](https://mlsanity.com/paper/2410.03659/) (2024)
- [Order Matters: Exploring Order Sensitivity in Multimodal Large Language Models](https://mlsanity.com/paper/2410.16983/) (2024)
- [Robust Multimodal Large Language Models Against Modality Conflict](https://mlsanity.com/paper/2507.07151/) (2025)
- [When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Models](https://mlsanity.com/paper/2508.15407/) (2025)
- [Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment](https://mlsanity.com/paper/2408.00137/) (2024)
- [Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict](https://mlsanity.com/paper/2504.08974/) (2025)
- [Do Images Speak Louder than Words? Investigating the Effect of Textual Misinformation in VLMs](https://mlsanity.com/paper/2601.19202/) (2026)
- [Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models](https://mlsanity.com/paper/2511.06168/) (2025)
- [Jailbreaking Vision-Language Models Through the Visual Modality](https://mlsanity.com/paper/2605.00583/) (2026)
- [PathMark: Protecting Intellectual Property of Mixture-of-Expert LLMs via Path Watermarks](https://mlsanity.com/paper/2607.03688/) (2026)
- [Do System Prompts Leave Behavioral Fingerprints? A Large-Scale Empirical Study of Clone Detection via Output Similarity](https://mlsanity.com/paper/2608.24461/) (2026)
- [FragileFlow: Spectral Control of Correct-but-Fragile Predictions for Foundation Model Robustness](https://mlsanity.com/paper/2605.08896/) (2026)
- [Rethinking Multi-Agent Intelligence Through the Lens of Small-World Networks](https://mlsanity.com/paper/2512.18094/) (2025)
- [Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Prompts](https://mlsanity.com/paper/2606.01441/) (2026)
- [SCOPE: Sequential Conformal Probing for Reliable OOD Rejection in LLM Services](https://mlsanity.com/paper/2606.21255/) (2026)

## Abstract (arXiv)

For multimodal large language models, when images or speech conflict with accompanying text, measured text reliance can entangle modality preference with evidence position. Earlier studies of text bias often used a fixed evidence order or moved task instructions with the evidence, leaving the contribution of order unclear. In this paper, we use a paired comparison that keeps the instructions and evidence content fixed and swaps only the positions of the two sources to quantify this potential influence. Across vision and speech models, placing an image or recording after conflicting text consistently shifts answers toward its content. We also revisit previous studies and analyze why their experimental settings can lead to misleading conclusions. These findings reveal cross-modal evidence noncommutativity: the same evidence can lead to different judgments when its order changes, and placing perceptual evidence later can increase the model's reliance on its content.