mlsanityData to 14 Oct 2026

SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature

arXiv 2601.10108 ↗First posted 15 Jan 2026cs.CL cs.AI cs.MMcited by 3

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Evaluating whether multimodal large language models truly understand long-form scientific papers remains challenging: answer-only metrics and synthetic "Needle-In-A-Haystack" tests often reward answer matching without requiring a causal, evidence-linked reasoning trace in the document. We propose the "Fish-in-the-Ocean" (FITO) paradigm, which requires models to construct explicit cross-modal evidence chains within native scientific documents. To operationalize FITO, we build SIN-Data, a scientific interleaved corpus that preserves the native interleaving of text and figures. On top of it, we construct SIN-Bench with four progressive tasks covering evidence discovery (SIN-Find), hypothesis verification (SIN-Verify), grounded QA (SIN-QA), and evidence-anchored synthesis (SIN-Summary). We further introduce "No Evidence, No Score", scoring predictions when grounded to verifiable anchors and diagnosing evidence quality via matching, relevance, and logic. Experiments on eight MLLMs show that grounding is the primary bottleneck: Gemini-3-pro achieves the best average overall score (0.573), while GPT-5 attains the highest SIN-QA answer accuracy (0.767) but underperforms on evidence-aligned overall scores, exposing a gap between correctness and traceable support.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 26 references that are in the map. First posted on arXiv 15 Jan 2026.

Builds on

15 of 26 references.

References · 15

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  5. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  6. 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
  7. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  8. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  9. 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 101
  10. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 72
  11. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
  12. 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Frameworkcited by 21
3 more
  1. 2024PIN: A Knowledge-Intensive Dataset for Paired and Interleaved Multimodal Documentscited by 7
  2. 2025Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmarkcited by 6
  3. 2025M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization?cited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering
  2. 2026From Papers to Mechanisms: An Evidence-Grounded Knowledge Substrate for Scientific Language Models
  3. 2025SciEGQA: A Dataset for Scientific Evidence-Grounded Question Answering and Reasoning
  4. 2025Can AI Validate Science? Benchmarking LLMs for Accurate Scientific Claim rightarrow Evidence Reasoning
  5. 2025Format Matters: The Robustness of Multimodal LLMs in Reviewing Evidence from Tables and Charts
  6. 2025DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections
  7. 2023Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning
  8. 2026BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidence