Investigating Relational Reasoning in VLMs
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Vision-Language Models (VLMs) achieve strong performance in visual reasoning tasks, but it remains unclear whether they understand visual relations, or simply employ shortcuts such as language cues or priors. To investigate this, we use the Qwen3-VL-4B (Bai et al., 2025), a modern VLM, to decode how visual information is encoded across depths. For this, we propose a synthetic dataset of simple geometric shapes for controlled analysis, along with queries crafted to precisely test language cues. Furthermore, the dataset is modified to test causal reliance on visual evidence. Our results show that current VLMs combine genuine visual reasoning with shortcut strategies primarily rooted in language cues.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 8 of its 13 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
8 of 13 references.
References · 8
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
- 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
- 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 153
- 2024Do Vision-Language Models Really Understand Visual Language?cited by 32
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens
- 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs
- 2024How Far Are We from Intelligent Visual Deductive Reasoning?
- 2025VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs
- 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
- 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
- 2024RelationVLM: Making Large Vision-Language Models Understand Visual Relations
- 2025Caption This, Reason That: VLMs Caught in the Middle