mlsanityData to 7 Oct 2026

DISSECT: Diagnosing Where Vision Ends and Language Priors Begin in Scientific VLMs

arXiv 2604.06250 ↗First posted 6 Apr 2026cs.CV cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

When asked to describe a molecular diagram, a Vision-Language Model correctly identifies ``a benzene ring with an -OH group.'' When asked to reason about the same image, it answers incorrectly. The model can see but it cannot think about what it sees. We term this the perception-integration gap: a failure where visual information is successfully extracted but lost during downstream reasoning, invisible to single-configuration benchmarks that conflate perception with integration under one accuracy number. To systematically expose such failures, we introduce DISSECT, a 12,000-question diagnostic benchmark spanning Chemistry (7,000) and Biology (5,000). Every question is evaluated under five input modes -- Vision+Text, Text-Only, Vision-Only, Human Oracle, and a novel Model Oracle in which the VLM first verbalizes the image and then reasons from its own description -- yielding diagnostic gaps that decompose performance into language-prior exploitation, visual extraction, perception fidelity, and integration effectiveness. Evaluating 18~VLMs, we find that: (1) Chemistry exhibits substantially lower language-prior exploitability than Biology, confirming molecular visual content as a harder test of genuine visual reasoning; (2) Open-source models consistently score higher when reasoning from their own verbalized descriptions than from raw images, exposing a systematic integration bottleneck; and (3) Closed-source models show no such gap, indicating that bridging perception and integration is the frontier separating open-source from closed-source multimodal capability. The Model Oracle protocol is both model and benchmark agnostic, applicable post-hoc to any VLM evaluation to diagnose integration failures.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 21 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

8 of 21 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025SciVerse: Unveiling the Knowledge Comprehension and Visual Reasoning of LMMs on Multi-modal Scientific Problems13 Mar
  2. 2024Probing Visual Language Priors in VLMs31 Dec
  3. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models15 Mar

Other references · 5

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  3. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  4. 2024ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Areacited by 68
  5. 2025Evaluating Large Language Models on Multimodal Chemistry Olympiad Examscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding
  2. 2026Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Act
  3. 2026Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning
  4. 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs
  5. 2026The Abstraction Gap in Vision-Language Causal Reasoning
  6. 2026MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding
  7. 2026PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
  8. 2025CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs