# Thinking with Tables: Enhancing Multi-Modal Tabular Understanding via Neuro-Symbolic Reasoning

arXiv: https://arxiv.org/abs/2603.24004 · HTML: https://mlsanity.com/paper/2603.24004/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 30 of its 62 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 30)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning](https://mlsanity.com/paper/2209.14610/) (2022)
- [LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL](https://mlsanity.com/paper/2503.07536/) (2025)
- [Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding](https://mlsanity.com/paper/2401.04398/) (2024)
- [Thyme: Think Beyond Images](https://mlsanity.com/paper/2508.11630/) (2025)
- [DeepAnalyze: Agentic Large Language Models for Autonomous Data Science](https://mlsanity.com/paper/2510.16872/) (2025)
- [Multimodal Table Understanding](https://mlsanity.com/paper/2406.08100/) (2024)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later](https://mlsanity.com/paper/2407.03257/) (2024)
- [StructLM: Towards Building Generalist Models for Structured Knowledge Grounding](https://mlsanity.com/paper/2402.16671/) (2024)
- [Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models](https://mlsanity.com/paper/2508.13678/) (2025)
- [Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning](https://mlsanity.com/paper/2506.01710/) (2025)
- [ChinaTravel: An Open-Ended Travel Planning Benchmark with Compositional Constraint Validation for Language Agents](https://mlsanity.com/paper/2412.13682/) (2024)
- [Skywork-R1V3 Technical Report](https://mlsanity.com/paper/2507.06167/) (2025)
- [Multimodal Tabular Reasoning with Privileged Structured Information](https://mlsanity.com/paper/2506.04088/) (2025)
- [TIME: TabPFN-Integrated Multimodal Engine for Robust Tabular-Image Learning](https://mlsanity.com/paper/2506.00813/) (2025)
- [LawGPT: Knowledge-Guided Data Generation and Its Application to Legal LLM](https://mlsanity.com/paper/2502.06572/) (2025)
- [HIPPO: Enhancing the Table Understanding Capability of LLMs through Hybrid-Modal Preference Optimization](https://mlsanity.com/paper/2502.17315/) (2025)
- [SADL: An Effective In-Context Learning Method for Compositional Visual QA](https://mlsanity.com/paper/2407.01983/) (2024)
- [TabularMath: Understanding Math Reasoning over Tables with Large Language Models](https://mlsanity.com/paper/2505.19563/) (2025)
- [TABLET: A Large-Scale Dataset for Robust Visual Table Understanding](https://mlsanity.com/paper/2509.21205/) (2025)

## Built on by (most cited first, 3 of 3)

- [Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning](https://mlsanity.com/paper/2608.07955/) (2026)
- [LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory](https://mlsanity.com/paper/2609.02350/) (2026)
- [SymboLLM-FE: LLM-Accelerated Symbolic Regression for Automated Feature Engineering on Tabular Data](https://mlsanity.com/paper/2608.28408/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated remarkable reasoning capabilities across modalities such as images and text. However, tabular data, despite being a critical real-world modality, remains relatively underexplored in multimodal learning. In this paper, we focus on the task of Tabular-Vision Multi-Modal Understanding (TVMU) and identify three core challenges: (1) high structural variability and data incompleteness in tables, (2) implicit and complex feature dependencies, and (3) significant heterogeneity in problem-solving pipelines across downstream tasks. To address these issues, we propose Thinking with Tables (TWT). TWT employs a program-aided code-based neuro-symbolic reasoning mechanism that facilitates key operations, such as information extraction and element modeling, by interacting with external environments. We evaluate TWT on eight representative datasets. Experimental results demonstrate that TWT consistently outperforms existing baselines by an average of 10% in accuracy, achieving performance comparable to, or even surpassing, proprietary commercial SOTA LLMs on TVMU tasks. Models and codes are available at https://github.com/kunyang-YU/Thinking-with-Tables