# MMCR: Benchmarking Cross-Source Reasoning in Scientific Papers

arXiv: https://arxiv.org/abs/2503.16856 · HTML: https://mlsanity.com/paper/2503.16856/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 27 of 27)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models](https://mlsanity.com/paper/2408.04840/) (2024)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://mlsanity.com/paper/2406.18521/) (2024)
- [CogVLM2: Visual Language Models for Image and Video Understanding](https://mlsanity.com/paper/2408.16500/) (2024)
- [InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output](https://mlsanity.com/paper/2407.03320/) (2024)
- [MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations](https://mlsanity.com/paper/2407.01523/) (2024)
- [TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document](https://mlsanity.com/paper/2403.04473/) (2024)
- [Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models](https://mlsanity.com/paper/2403.00231/) (2024)
- [Efficient Multimodal Learning from Data-centric Perspective](https://mlsanity.com/paper/2402.11530/) (2024)
- [mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding](https://mlsanity.com/paper/2409.03420/) (2024)
- [MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding](https://mlsanity.com/paper/2407.04903/) (2024)
- [LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models](https://mlsanity.com/paper/2407.19185/) (2024)
- [Rethinking Comprehensive Benchmark for Chart Understanding: A Perspective from Scientific Literature](https://mlsanity.com/paper/2412.12150/) (2024)

## Built on by (most cited first, 5 of 5)

- [PaperMind: Benchmarking Agentic Reasoning and Critique over Scientific Papers in Multimodal LLMs](https://mlsanity.com/paper/2604.21304/) (2026)
- [DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agents](https://mlsanity.com/paper/2606.26122/) (2026)
- [Benchmarking PhD-Level Coding in 3D Geometric Computer Vision](https://mlsanity.com/paper/2603.30038/) (2026)
- [PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers](https://mlsanity.com/paper/2604.11307/) (2026)
- [ResearchArcade: Graph Interface for Academic Tasks](https://mlsanity.com/paper/2511.22036/) (2025)

## Abstract (arXiv)

Fully comprehending scientific papers by machines reflects a high level of Artificial General Intelligence, requiring the ability to reason across fragmented and heterogeneous sources of information, presenting a complex and practically significant challenge. While Vision-Language Models (VLMs) have made remarkable strides in various tasks, particularly those involving reasoning with evidence source from single image or text page, their ability to use cross-source information for reasoning remains an open problem. This work presents MMCR, a high-difficulty benchmark designed to evaluate VLMs' capacity for reasoning with cross-source information from scientific papers. The benchmark comprises 276 high-quality questions, meticulously annotated by humans across 7 subjects and 10 task types. Experiments with 18 VLMs demonstrate that cross-source reasoning presents a substantial challenge for existing models. Notably, even the top-performing model, GPT-4o, achieved only 48.55% overall accuracy, with only 20% accuracy in multi-table comprehension tasks, while the second-best model, Qwen2.5-VL-72B, reached 39.86% overall accuracy. Furthermore, we investigated the impact of the Chain-of-Thought (CoT) technique on cross-source reasoning and observed a detrimental effect on small models, whereas larger models demonstrated substantially enhanced performance. These results highlight the pressing need to develop VLMs capable of effectively utilizing cross-source information for reasoning.