mlsanityData to 13 Oct 2026

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

arXiv 2609.11243 ↗First posted 10 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Autonomous research agents are increasingly expected to search the literature, analyze experimental evidence, and generate scientific hypotheses. These capabilities require multi-step evidence grounded reasoning that progressively acquires, integrates, and verifies evidence before reaching a conclusion. Existing multimodal benchmarks, however, largely evaluate final-answer accuracy, leaving open whether predictions are actually supported by traceable scientific evidence. We introduce Sci-MMR, a benchmark for multi-step evidence-grounded scientific reasoning built on structured argument graphs linking scientific claims, citation-grounded knowledge, visual evidence, and supporting regions. Sci-MMR comprises 235 multi-hop reasoning tasks spanning four scientific disciplines, with an average of nine figure panels per task. Evaluating eight frontier multimodal models, we find that answer accuracy consistently exceeds complete-evidence recovery rate by more than 20%, revealing a substantial gap that answer-only evaluation is structurally unable to capture. Through controlled interventions, we identify two fundamental bottlenecks. First, evidence acquisition: models struggle to extract complete structured evidence from scientific figures, accounting for 57.2% of failures. While cropping tools yield modest gains (+4.5 points), providing gold evidence improves accuracy by up to 37.0 points, indicating difficulty in assembling complete multi-region evidence. Second, evidence integration: models struggle to translate available evidence into correct conclusions, accounting for 31.8% of failures, while even with gold evidence the strongest model achieves only 69.1% accuracy on the hardest tasks. These findings indicate that current answer-centric benchmarks substantially overestimate the evidence-grounded reasoning capabilities of multimodal research agents

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 37 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

15 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains1 Mar

Other references · 14

  1. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,041
  2. 2025Accelerating scientific discovery with Co-Scientistcited by 383
  3. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 258
  4. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 224
  5. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agentscited by 81
  6. 2026GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agentscited by 32
  7. 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Modelscited by 31
  8. 2026MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agentscited by 22
  9. 2026Towards Long-horizon Agentic Multimodal Searchcited by 10
  10. 2026SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agentscited by 9
  11. 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcomecited by 6
  12. 2026HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoningcited by 5
2 more
  1. 2026CRIT: Graph-Based Automatic Data Synthesis to Enhance Cross-Modal Multi-Hop Reasoningcited by 2
  2. 2026M2-Verify: A Large-Scale Multidomain Benchmark for Checking Multimodal Claim Consistencycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation
  2. 2026PaperMind: Benchmarking Agentic Reasoning and Critique over Scientific Papers in Multimodal LLMs
  3. 2025Can AI Validate Science? Benchmarking LLMs for Accurate Scientific Claim rightarrow Evidence Reasoning
  4. 2026SciR: A Controllable Benchmark for Scientific Reasoning in LLMs
  5. 2026PathAnchor: Path-Structured Evidence for Scientific Agents
  6. 2026SCIRIGOR:Evaluating Open-Ended Scientific Analysis Beyond Final Scores
  7. 2026Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents
  8. 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning