mlsanityData to 14 Oct 2026

ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning

arXiv 2607.16131 ↗First posted 17 Jul 2026cs.CL cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Multimodal Scientific Claim Verification (MSCV) requires models to verify scientific claims using visually grounded evidence from papers, including figures, tables, charts, and textual context. However, existing methods often fail because they struggle to locate decisive visual evidence, accurately read structured scientific visuals, and integrate multimodal observations into reliable reasoning. We introduce ToolSciVer, the first tool-augmented framework for MSCV to our knowledge. ToolSciVer equips a VLM with three type-aware visual tools, table row/column focus, chart-to-structure parsing, and high-resolution region zoom, which convert dense scientific visuals into explicit, claim-facing evidence, and trains the policy with Group Relative Policy Optimization (GRPO) under a composite reward of answer correctness, format validity, length control, tool-use efficiency, and tool-validity penalties. Experiments on SciVer and MuSciClaims datasets on five VLMs from three model families (Qwen, InternVL, Gemma) demonstrate that our method achieves superior performance compared to four competitive baselines including prompting-based and RL-based tool-use methods, highlighting the effectiveness of learned, type-aware tool use for scientific claim verification.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 36 references that are in the map. First posted on arXiv 17 Jul 2026.

Builds on

32 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May

Other references · 31

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  8. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  9. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  10. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  11. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  12. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
19 more
  1. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  2. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  3. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
  4. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  5. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 268
  6. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
  7. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 133
  8. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 96
  9. 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
  10. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Usecited by 90
  11. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understandingcited by 83
  12. 2025Acting Less is Reasoning More! Teaching Model to Act Efficientlycited by 80
  13. 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tablescited by 51
  14. 2025SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verificationcited by 17
  15. 2024Piecing It All Together: Verifying Multi-Hop Multimodal Claimscited by 12
  16. 2025MuSciClaims: Multimodal Scientific Claim Verificationcited by 8
  17. 2026SciClaimEval: Cross-modal Claim Verification in Scientific Paperscited by 7
  18. 2025MultiCheck: Strengthening Web Trust with Unified Multimodal Fact Verificationcited by 1
  19. 2025Format Matters: The Robustness of Multimodal LLMs in Reviewing Evidence from Tables and Chartscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Not Just Reason, Not Just Scan: Reinforcement Learning for Proactive Scientific Error Verification over Academic Paper
  2. 2026SciLens: Multi-modal Scientific Claim Verification with Agentic Entailment and Grounding
  3. 2026Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents
  4. 2026SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task
  5. 2026Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessity
  6. 2025Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning
  7. 2026V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
  8. 2026Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning