mlsanityData to 7 Oct 2026

TableBench: A Comprehensive and Complex Benchmark for Table Question Answering

arXiv 2408.09174 ↗First posted 17 Aug 2024cs.CLcited by 127

Not on the map. Why?

Cited by 32 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in Large Language Models (LLMs) have markedly enhanced the interpretation and processing of tabular data, introducing previously unimaginable capabilities. Despite these achievements, LLMs still encounter significant challenges when applied in industrial scenarios, particularly due to the increased complexity of reasoning required with real-world tabular data, underscoring a notable disparity between academic benchmarks and practical applications. To address this discrepancy, we conduct a detailed investigation into the application of tabular data in industrial scenarios and propose a comprehensive and complex benchmark TableBench, including 18 fields within four major categories of table question answering (TableQA) capabilities. Furthermore, we introduce TableLLM, trained on our meticulously constructed training set TableInstruct, achieving comparable performance with GPT-3.5. Massive experiments conducted on TableBench indicate that both open-source and proprietary LLMs still have significant room for improvement to meet real-world demands, where the most advanced model, GPT-4, achieves only a modest score compared to humans.

Key facts. Cited by 127 papers in the map (data through 7 Oct 2026). Builds on 40 of its 55 references that are in the map. First posted on arXiv 17 Aug 2024.

Builds on

40 of 55 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2024Qwen2 Technical Reportcited by 2,076
  7. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  10. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  11. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  12. 2024Yi: Open Foundation Models by 01.AIcited by 706
28 more
  1. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  2. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  3. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  4. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  5. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  6. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  7. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  8. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  9. 2020Open Question Answering over Tables and Textcited by 163
  10. 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQLcited by 158
  11. 2024TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarioscited by 76
  12. 2023Exploring Chain-of-Thought Style Prompting for Text-to-SQLcited by 69
  13. 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
  14. 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Seriescited by 67
  15. 2023TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPTcited by 57
  16. 2022The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checkingcited by 56
  17. 2024Large Language Model for Table Processing: A Surveycited by 52
  18. 2023Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMscited by 49
  19. 2024xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoningcited by 46
  20. 2021AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industrycited by 44
  21. 2022A Survey on Table Question Answering: Recent Advancescited by 41
  22. 2023LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzlescited by 41
  23. 2023Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queriescited by 38
  24. 2024StructLM: Towards Building Generalist Models for Structured Knowledge Groundingcited by 29
  25. 2023Large Language Models are Complex Table Parserscited by 28
  26. 2019Answering Conversational Questions on Structured Data without Logical Formscited by 24
  27. 2024When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Modelscited by 22
  28. 2023Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characterscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 64; 2026: 54.

127 papers.

Sort:

By area:

  1. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  2. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 69
  3. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  4. 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 61
  5. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Searchcited by 53
  6. 2025Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Trainingcited by 42
  7. 2025Griffin: Towards a Graph-Centric Relational Database Foundation Modelcited by 41
  8. 2025Table-R1: Region-based Reinforcement Learning for Table Understandingcited by 35
  9. 2025Steering LLM Thinking with Budget Guidancecited by 30
  10. 2024Evaluating and Aligning CodeLLMs on Human Preferencecited by 22
  11. 2025nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflowcited by 21
  12. 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understandingcited by 20
38 more
  1. 2025VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Modelscited by 19
  2. 2025SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Mannercited by 18
  3. 2024TQA-Bench: Evaluating LLMs for Multi-Table Question Answeringcited by 17
  4. 2025TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answeringcited by 14
  5. 2025MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmarkcited by 13
  6. 2025Not All Parameters Are Created Equal: Smart Isolation Boosts Fine-Tuning Performancecited by 13
  7. 2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skillscited by 13
  8. 2024AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Frameworkcited by 12
  9. 2025RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysiscited by 12
  10. 2025Scaling Generalist Data-Analytic Agentscited by 11
  11. 2025Structural Reward Model: Enhancing Interpretability, Efficiency, and Scalability in Reward Modelingcited by 11
  12. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  13. 2024SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Typescited by 10
  14. 2025Rethinking Table Instruction Tuningcited by 10
  15. 2025RoT: Enhancing Table Reasoning with Iterative Row-Wise Traversalscited by 10
  16. 2025MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Spacecited by 9
  17. 2025A Survey of AI Scientistscited by 9
  18. 2024Large Language Models Engineer Too Many Simple Features For Tabular Datacited by 8
  19. 2025NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tablescited by 8
  20. 2025FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Financecited by 8
  21. 2025StructCoh: Structured Contrastive Learning for Context-Aware Text Semantic Matchingcited by 8
  22. 2026Rethinking Table Pruning in TableQA: From Sequential Revisions to Gold Trajectory-Supervised Parallel Searchcited by 8
  23. 2025General Table Question Answering via Answer-Formula Joint Generationcited by 7
  24. 2025Plugging Schema Graph into Multi-Table QA: A Human-Guided Framework for Reducing LLM Reliancecited by 7
  25. 2025TableGPT-R1: Advancing Tabular Reasoning Through Reinforcement Learningcited by 7
  26. 2026Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Modelscited by 7
  27. 2026SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflowscited by 7
  28. 2025CoddLLM: Empowering Large Language Models for Data Analyticscited by 6
  29. 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answeringcited by 6
  30. 2025Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Studycited by 6
  31. 2025Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understandingcited by 6
  32. 2026FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agentscited by 6
  33. 2025Text-to-Pipeline: Bridging Natural Language and Data Preparation Pipelinescited by 5
  34. 2025IFEvalCode: Controlled Code Generationcited by 5
  35. 2025T2R-bench: A Benchmark for Generating Article-Level Reports from Real World Industrial Tablescited by 5
  36. 2025TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoningcited by 5
  37. 2025TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Datacited by 5
  38. 2025TeleTables: A Benchmark for Large Language Models in Telecom Table Interpretationcited by 5

50 most cited of 127 All 127 (JSON)