Don't Look at the Numbers: Visual Anchoring Bias and Layer-wise Representation in VLMs
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Embedded numeric anchors on images systematically bias Vision-Language Model quality judgments across six VLMs from five architectural families (ANOVA eta^2 = 0.18-0.77, all p < 0.001). Anchor effects are 2.5x larger than severe image quality degradation, confirming bias is not reducible to visual changes. Layer-wise probing reveals consistent dissociation: layers where anchor classification saturates (L12-L34) are suboptimal for quality prediction, with optimal layers deeper (R^2 = 0.69-0.91). Fusion analysis identifies architecture-dependent integration -- instant fusion at L1-L2 in two models versus partial or no fusion in three others. These results establish a causal account of visual anchoring bias, linking behavioral susceptibility to representation dynamics.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 11 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
8 of 11 references.
Closest ancestors · 1
Same direction, newest first
- 2024Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Image22 Feb
Other references · 7
- 2024Cognitive Bias in Decision-Making with LLMscited by 106
- 2024Anchoring Bias in Large Language Models: An Experimental Studycited by 37
- 2025Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigationcited by 23
- 2025Vision Function Layer in Multimodal LLMscited by 17
- 2025Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Modelscited by 14
- 2025Dyslexify: A Mechanistic Defense Against Typographic Attacks in CLIPcited by 9
- 2026Where Vision Becomes Text: Locating the OCR Routing Bottleneck in Vision-Language Modelscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception
- 2025Demystifying the Visual Quality Paradox in Multimodal Large Language Models
- 2026V-FAT: Benchmarking Visual Fidelity Against Text-bias
- 2026Same Answer, Different Representations: Hidden instability in VLMs
- 2026Scaling Vision-Language Models Is Not Enough to Mitigate Bias
- 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict
- 2026Can't Find Waldo: Evaluating VLMs' Sensitivity to Image Resolution and Detail Level
- 2025On the dynamic evolution of CLIP texture-shape bias and its relationship to human alignment and model robustness