# Does Table Source Matter? Benchmarking and Improving Multimodal Scientific Table Understanding and Reasoning

arXiv: https://arxiv.org/abs/2501.13042 · HTML: https://mlsanity.com/paper/2501.13042/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 52 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [Pixtral 12B](https://mlsanity.com/paper/2410.07073/) (2024)
- [TableLlama: Towards Open Large Generalist Models for Tables](https://mlsanity.com/paper/2311.09206/) (2023)
- [Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study](https://mlsanity.com/paper/2305.13062/) (2023)
- [TableGPT2: A Large Multimodal Model with Tabular Data Integration](https://mlsanity.com/paper/2411.02059/) (2024)
- [Table-GPT: Table-tuned GPT for Diverse Table Tasks](https://mlsanity.com/paper/2310.09263/) (2023)

## Built on by (most cited first, 15 of 15)

- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [Multimodal Tabular Reasoning with Privileged Structured Information](https://mlsanity.com/paper/2506.04088/) (2025)
- [Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision](https://mlsanity.com/paper/2503.01879/) (2025)
- [Can GRPO Boost Complex Multimodal Table Understanding?](https://mlsanity.com/paper/2509.16889/) (2025)
- [TableMoE: Neuro-Symbolic Routing for Structured Expert Reasoning in Multimodal Table Understanding](https://mlsanity.com/paper/2506.21393/) (2025)
- [SciClaimEval: Cross-modal Claim Verification in Scientific Papers](https://mlsanity.com/paper/2602.07621/) (2026)
- [Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation](https://mlsanity.com/paper/2510.09671/) (2025)
- [Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers](https://mlsanity.com/paper/2506.10486/) (2025)
- [Analyzing LLM Instruction Optimization for Tabular Fact Verification](https://mlsanity.com/paper/2602.17937/) (2026)
- [TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexity](https://mlsanity.com/paper/2605.05955/) (2026)
- [TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs](https://mlsanity.com/paper/2606.09578/) (2026)
- [Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder](https://mlsanity.com/paper/2506.20083/) (2025)
- [An Empirical Investigation of Robustness in Large Language Models under Tabular Distortions](https://mlsanity.com/paper/2601.05009/) (2026)
- [InterTab: Interleaved Visual-Structure Alignment for Multi-Modal Table Reasoning](https://mlsanity.com/paper/2609.32660/) (2026)
- [RxnOptBench: Benchmarking LLMs for Reaction-Condition Optimization in Organic Methodology](https://mlsanity.com/paper/2610.02242/) (2026)

## Abstract (arXiv)

Recent large language models (LLMs) have advanced table understanding capabilities but rely on converting tables into text sequences. While multimodal large language models (MLLMs) enable direct visual processing, they face limitations in handling scientific tables due to fixed input image resolutions and insufficient numerical reasoning capabilities. We present a comprehensive framework for multimodal scientific table understanding and reasoning with dynamic input image resolutions. Our framework consists of three key components: (1) MMSci-Pre, a domain-specific table structure learning dataset of 52K scientific table structure recognition samples, (2) MMSci-Ins, an instruction tuning dataset with 12K samples across three table-based tasks, and (3) MMSci-Eval, a benchmark with 3,114 testing samples specifically designed to evaluate numerical reasoning capabilities. Extensive experiments demonstrate that our domain-specific approach with 52K scientific table images achieves superior performance compared to 150K general-domain tables, highlighting the importance of data quality over quantity. Our proposed table-based MLLMs with dynamic input resolutions show significant improvements in both general table understanding and numerical reasoning capabilities, with strong generalisation to held-out datasets. Our code and data are publicly available at https://github.com/Bernard-Yang/MMSci_Table.