mlsanityData to 14 Oct 2026

Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning

arXiv 2604.27604 ↗First posted 30 Apr 2026cs.CV cs.CEcited by 1

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs' ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 42 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

6 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination15 Nov
  2. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization15 Nov

Other references · 4

  1. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
  2. 2023SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphscited by 68
  3. 2025A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applicationscited by 50
  4. 2024SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Modelscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SciFigQual-Bench: A Benchmark for Scientific Figure Quality Assessment with Full-Manuscript Contextcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs
  2. 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
  3. 2025VER-Bench: Evaluating MLLMs on Reasoning with Fine-Grained Visual Evidence
  4. 2025SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models
  5. 2026SIQA: Toward Reliable Scientific Image Quality Assessment
  6. 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
  7. 2025MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research
  8. 2026Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark