# TableBench: A Comprehensive and Complex Benchmark for Table Question Answering

arXiv: https://arxiv.org/abs/2408.09174 · HTML: https://mlsanity.com/paper/2408.09174/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 55 references that are in the map.
- First posted on arXiv 17 Aug 2024.

## Builds on (most cited first, 30 of 40)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [FeTaQA: Free-form Table Question Answering](https://mlsanity.com/paper/2104.00369/) (2021)
- [MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL](https://mlsanity.com/paper/2312.11242/) (2023)
- [Open Question Answering over Tables and Text](https://mlsanity.com/paper/2010.10439/) (2020)
- [TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios](https://mlsanity.com/paper/2403.19318/) (2024)
- [Exploring Chain-of-Thought Style Prompting for Text-to-SQL](https://mlsanity.com/paper/2305.14215/) (2023)
- [Table-GPT: Table-tuned GPT for Diverse Table Tasks](https://mlsanity.com/paper/2310.09263/) (2023)
- [MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series](https://mlsanity.com/paper/2405.19327/) (2024)
- [TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPT](https://mlsanity.com/paper/2307.08674/) (2023)
- [The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking](https://mlsanity.com/paper/2203.00991/) (2022)
- [Large Language Model for Table Processing: A Survey](https://mlsanity.com/paper/2402.05121/) (2024)
- [Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMs](https://mlsanity.com/paper/2310.10358/) (2023)

## Built on by (most cited first, 30 of 128)

- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [TableGPT2: A Large Multimodal Model with Tabular Data Integration](https://mlsanity.com/paper/2411.02059/) (2024)
- [SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models](https://mlsanity.com/paper/2502.13059/) (2025)
- [Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent](https://mlsanity.com/paper/2411.02937/) (2024)
- [Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search](https://mlsanity.com/paper/2502.02508/) (2025)
- [Griffin: Towards a Graph-Centric Relational Database Foundation Model](https://mlsanity.com/paper/2505.05568/) (2025)
- [Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training](https://mlsanity.com/paper/2508.00414/) (2025)
- [Table-R1: Region-based Reinforcement Learning for Table Understanding](https://mlsanity.com/paper/2505.12415/) (2025)
- [Steering LLM Thinking with Budget Guidance](https://mlsanity.com/paper/2506.13752/) (2025)
- [nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow](https://mlsanity.com/paper/2502.05036/) (2025)
- [Evaluating and Aligning CodeLLMs on Human Preference](https://mlsanity.com/paper/2412.05210/) (2024)
- [Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding](https://mlsanity.com/paper/2503.14140/) (2025)
- [VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Models](https://mlsanity.com/paper/2505.15801/) (2025)
- [SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner](https://mlsanity.com/paper/2506.09003/) (2025)
- [Terminal-World: Scaling Terminal-Agent Environments via Agent Skills](https://mlsanity.com/paper/2605.20876/) (2026)
- [Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?](https://mlsanity.com/paper/2607.09996/) (2026)
- [TQA-Bench: Evaluating LLMs for Multi-Table Question Answering](https://mlsanity.com/paper/2411.19504/) (2024)
- [TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answering](https://mlsanity.com/paper/2506.03949/) (2025)
- [MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2506.05587/) (2025)
- [Not All Parameters Are Created Equal: Smart Isolation Boosts Fine-Tuning Performance](https://mlsanity.com/paper/2508.21741/) (2025)
- [AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Framework](https://mlsanity.com/paper/2412.10422/) (2024)
- [RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysis](https://mlsanity.com/paper/2506.13405/) (2025)
- [Scaling Generalist Data-Analytic Agents](https://mlsanity.com/paper/2509.25084/) (2025)
- [Structural Reward Model: Enhancing Interpretability, Efficiency, and Scalability in Reward Modeling](https://mlsanity.com/paper/2509.25361/) (2025)
- [A Survey of AI Scientists](https://mlsanity.com/paper/2510.23045/) (2025)
- [SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows](https://mlsanity.com/paper/2606.29955/) (2026)
- [Large Language Models Engineer Too Many Simple Features For Tabular Data](https://mlsanity.com/paper/2410.17787/) (2024)
- [SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types](https://mlsanity.com/paper/2412.11757/) (2024)
- [Rethinking Table Instruction Tuning](https://mlsanity.com/paper/2501.14693/) (2025)
- [RoT: Enhancing Table Reasoning with Iterative Row-Wise Traversals](https://mlsanity.com/paper/2505.15110/) (2025)

## Abstract (arXiv)

Recent advancements in Large Language Models (LLMs) have markedly enhanced the interpretation and processing of tabular data, introducing previously unimaginable capabilities. Despite these achievements, LLMs still encounter significant challenges when applied in industrial scenarios, particularly due to the increased complexity of reasoning required with real-world tabular data, underscoring a notable disparity between academic benchmarks and practical applications. To address this discrepancy, we conduct a detailed investigation into the application of tabular data in industrial scenarios and propose a comprehensive and complex benchmark TableBench, including 18 fields within four major categories of table question answering (TableQA) capabilities. Furthermore, we introduce TableLLM, trained on our meticulously constructed training set TableInstruct, achieving comparable performance with GPT-3.5. Massive experiments conducted on TableBench indicate that both open-source and proprietary LLMs still have significant room for improvement to meet real-world demands, where the most advanced model, GPT-4, achieves only a modest score compared to humans.