mlsanityData to 14 Oct 2026

MMTABREAL: Real-World Benchmark for Multimodal Table Understanding

arXiv 2505.21771 ↗First posted 27 May 2025cs.CV cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Multimodal tables i.e. tabular layouts interleaved with charts, maps, icons, and color encodings are ubiquitous in real applications yet remain difficult for Multimodal Large Language Models (MLLMs). Despite advances in text and image understanding, systematic evaluation of table-centric multimodal reasoning is limited. We introduce MMTABREAL, a MultiModal Table Benchmark, human-curated suite of 500 real-world tables paired with 4,021 question-answer pairs. MMTABREAL spans four question types, five reasoning categories, and eight structural archetypes. Evaluations of state-of-the-art models reveal substantial gaps, especially in visual grounding, spatial alignment, and multi-step inference, with 20-40% performance drops relative to existing benchmarks. These results highlight the need for architectures that more tightly fuse vision with tabular structure and support explicit numeric/logical operations. MMTABREAL is released for evaluation only, providing a rigorous, reproducible testbed that reflects the linguistic, structural, and reasoning complexity of real-world multimodal tables.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

27 of 53 references.

References · 27

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  3. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  5. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  6. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  7. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  8. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  9. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  10. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 264
  11. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  12. 2024Rotary Position Embedding for Vision Transformercited by 236
15 more
  1. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 216
  2. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  3. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 179
  4. 2021FeTaQA: Free-form Table Question Answeringcited by 175
  5. 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 159
  6. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuningcited by 150
  7. 2024What makes unlearning hard and what to do about itcited by 89
  8. 2024CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacycited by 81
  9. 2023Open-WikiTable: Dataset for Open Domain Question Answering with Complex Reasoning over Tablecited by 40
  10. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  11. 2022SPIQ: Data-Free Per-Channel Static Input Quantizationcited by 12
  12. 2024mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoningcited by 9
  13. 2024Knowledge-Aware Reasoning over Multimodal Semi-structured Tablescited by 9
  14. 2024KET-QA: A Dataset for Knowledge Enhanced Table Question Answeringcited by 5
  15. 2021An Algorithm to Warm Start Perturbed (WASP) Constrained Dynamic Programscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 7.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs
  2. 2024Multimodal Table Understanding
  3. 2025Multimodal Tabular Reasoning with Privileged Structured Information
  4. 2026Thinking with Tables: Enhancing Multi-Modal Tabular Understanding via Neuro-Symbolic Reasoning
  5. 2025MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space
  6. 2025TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding
  7. 2026CFMS: A Coarse-to-Fine Multimodal Synthesis Framework for Enhanced Tabular Reasoning
  8. 2025Visual-TableQA: Open-Domain Benchmark for Reasoning over Table Images