mlsanityData to 7 Oct 2026

Multimodal Tabular Reasoning with Privileged Structured Information

arXiv 2506.04088 ↗First posted 4 Jun 2025cs.LG cs.AI cs.CLcited by 14

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Tabular reasoning involves multi-step information extraction and logical inference over tabular data. While recent advances have leveraged large language models (LLMs) for reasoning over structured tables, such high-quality textual representations are often unavailable in real-world settings, where tables typically appear as images. In this paper, we tackle the task of tabular reasoning from table images, leveraging privileged structured information available during training to enhance multimodal large language models (MLLMs). The key challenges lie in the complexity of accurately aligning structured information with visual representations, and in effectively transferring structured reasoning skills to MLLMs despite the input modality gap. To address these, we introduce TabUlar Reasoning with Bridged infOrmation (sc Turbo), a new framework for multimodal tabular reasoning with privileged structured tables. sc Turbo benefits from a structure-aware reasoning trace generator based on DeepSeek-R1, contributing to high-quality modality-bridged data. On this basis, sc Turbo repeatedly generates and selects the advantageous reasoning paths, further enhancing the model's tabular reasoning ability. Experimental results demonstrate that, with limited (9k) data, sc Turbo achieves state-of-the-art performance (+7.2% vs. previous SOTA) across multiple datasets.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 65 of its 93 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

65 of 93 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Does Table Source Matter? Benchmarking and Improving Multimodal Scientific Table Understanding and Reasoning22 Jan
  2. 2024Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs19 Feb
  3. 2024A Survey of Table Reasoning with Large Language Models13 Feb
  4. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding9 Jan
  5. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning31 Jan
  6. 2021HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation15 Aug

Other references · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2025Qwen2.5-VL Technical Reportcited by 5,418
38 more
  1. 2024Qwen2.5 Technical Reportcited by 4,713
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  7. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  8. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  9. 2020A Survey on Visual Transformercited by 653
  10. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  11. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  12. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  13. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  14. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  15. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  16. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  17. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  18. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  19. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 167
  20. 2020Open Question Answering over Tables and Textcited by 163
  21. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Modelscited by 133
  22. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  23. 2019TableBank: A Benchmark Dataset for Table Detection and Recognitioncited by 102
  24. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  25. 2022TableFormer: Table Structure Understanding with Transformerscited by 93
  26. 2020INFOTABS: Inference on Tables as Semi-structured Datacited by 84
  27. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 75
  28. 2023UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understandingcited by 74
  29. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  30. 2024TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergycited by 70
  31. 2020TableNet: Deep Learning model for end-to-end Table detection and Tabular data extraction from Scanned Document Imagescited by 69
  32. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 69
  33. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  34. 2024TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domainscited by 68
  35. 2024Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoningcited by 52
  36. 2024TALENT: A Tabular Analytics and Learning Toolboxcited by 46
  37. 2025Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuningcited by 44
  38. 2024Multimodal Table Understandingcited by 43

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 9.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026CFMS: A Coarse-to-Fine Multimodal Synthesis Framework for Enhanced Tabular Reasoning
  2. 2024Knowledge-Aware Reasoning over Multimodal Semi-structured Tables
  3. 2025MMTABREAL: Real-World Benchmark for Multimodal Table Understanding
  4. 2026InterTab: Interleaved Visual-Structure Alignment for Multi-Modal Table Reasoning
  5. 2026TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs
  6. 2023Rethinking Tabular Data Understanding with Large Language Models
  7. 2025Visual-TableQA: Open-Domain Benchmark for Reasoning over Table Images
  8. 2026TABQAWORLD: Optimizing Multimodal Reasoning for Multi-Turn Table Question Answering