# TableRAG: Million-Token Table Understanding with Language Models

arXiv: https://arxiv.org/abs/2410.04739 · HTML: https://mlsanity.com/paper/2410.04739/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [Binding Language Models in Symbolic Languages](https://mlsanity.com/paper/2210.02875/) (2022)
- [Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding](https://mlsanity.com/paper/2401.04398/) (2024)
- [Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning](https://mlsanity.com/paper/2301.13808/) (2023)
- [Query2doc: Query Expansion with Large Language Models](https://mlsanity.com/paper/2303.07678/) (2023)
- [TableLlama: Towards Open Large Generalist Models for Tables](https://mlsanity.com/paper/2311.09206/) (2023)
- [Evaluating the Text-to-SQL Capabilities of Large Language Models](https://mlsanity.com/paper/2204.00498/) (2022)
- [Natural Language to Code Generation in Interactive Data Science Notebooks](https://mlsanity.com/paper/2212.09248/) (2022)
- [ReAcTable: Enhancing ReAct for Table Question Answering](https://mlsanity.com/paper/2310.00815/) (2023)
- [Understanding tables with intermediate pre-training](https://mlsanity.com/paper/2010.00571/) (2020)
- [DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models](https://mlsanity.com/paper/2402.01117/) (2024)
- [Rethinking Tabular Data Understanding with Large Language Models](https://mlsanity.com/paper/2312.16702/) (2023)
- [MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering](https://mlsanity.com/paper/2305.12820/) (2023)
- [TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning](https://mlsanity.com/paper/2312.09039/) (2023)

## Built on by (most cited first, 30 of 41)

- [Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models](https://mlsanity.com/paper/2503.01763/) (2025)
- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents](https://mlsanity.com/paper/2505.19549/) (2025)
- [LLM-Forest: Ensemble Learning of LLMs with Graph-Augmented Prompts for Data Imputation](https://mlsanity.com/paper/2410.21520/) (2024)
- [Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understanding](https://mlsanity.com/paper/2411.08516/) (2024)
- [TQA-Bench: Evaluating LLMs for Multi-Table Question Answering](https://mlsanity.com/paper/2411.19504/) (2024)
- [TARGET: Benchmarking Table Retrieval for Generative Tasks](https://mlsanity.com/paper/2505.11545/) (2025)
- [AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding](https://mlsanity.com/paper/2506.13589/) (2025)
- [Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQL](https://mlsanity.com/paper/2504.15077/) (2025)
- [When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tables](https://mlsanity.com/paper/2509.17680/) (2025)
- [NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables](https://mlsanity.com/paper/2504.06560/) (2025)
- [Rethinking Table Pruning in TableQA: From Sequential Revisions to Gold Trajectory-Supervised Parallel Search](https://mlsanity.com/paper/2601.03851/) (2026)
- [Chain-of-Query: Unleashing the Power of LLMs in SQL-Aided Table Understanding via Multi-Agent Collaboration](https://mlsanity.com/paper/2508.15809/) (2025)
- [Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation](https://mlsanity.com/paper/2510.09671/) (2025)
- [Beyond Relational: Semantic-Aware Multi-Modal Analytics with LLM-Native Query Optimization](https://mlsanity.com/paper/2511.19830/) (2025)
- [AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents](https://mlsanity.com/paper/2510.00039/) (2025)
- [How Good Are LLMs at Processing Tool Outputs?](https://mlsanity.com/paper/2510.15955/) (2025)
- [Improving Table Understanding with LLMs and Entity-Oriented Search](https://mlsanity.com/paper/2508.17028/) (2025)
- [TableZoomer: A Collaborative Agent Framework for Large-scale Table Question Answering](https://mlsanity.com/paper/2509.01312/) (2025)
- [Same Content, Different Representations: A Controlled Study for Table QA](https://mlsanity.com/paper/2509.22983/) (2025)
- [Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Models](https://mlsanity.com/paper/2601.11342/) (2026)
- [Towards Question Answering over Large Semi-structured Tables](https://mlsanity.com/paper/2502.13422/) (2025)
- [MRT at SemEval-2025 Task 8: Maximizing Recovery from Tables with Multiple Steps](https://mlsanity.com/paper/2505.22264/) (2025)
- [Toward Real-World Table Agents: Capabilities, Workflows, and Design Principles for LLM-based Table Intelligence](https://mlsanity.com/paper/2507.10281/) (2025)
- [Towards Probabilistic Question Answering Over Tabular Data](https://mlsanity.com/paper/2506.20747/) (2025)
- [What to Keep and What to Drop: Adaptive Table Filtering Framework](https://mlsanity.com/paper/2506.23463/) (2025)
- [An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems](https://mlsanity.com/paper/2501.00562/) (2024)
- [Evaluating VisualRAG: Quantifying Cross-Modal Performance in Enterprise Document Understanding](https://mlsanity.com/paper/2506.21604/) (2025)
- [MRT at IberLEF-2025 PRESTA Task: Maximizing Recovery from Tables with Multiple Steps](https://mlsanity.com/paper/2507.12981/) (2025)
- [Scan-do Attitude: Towards Autonomous CT Protocol Management using a Large Language Model Agent](https://mlsanity.com/paper/2509.20270/) (2025)

## Abstract (arXiv)

Recent advancements in language models (LMs) have notably enhanced their ability to reason with tabular data, primarily through program-aided mechanisms that manipulate and analyze tables. However, these methods often require the entire table as input, leading to scalability challenges due to the positional bias or context length constraints. In response to these challenges, we introduce TableRAG, a Retrieval-Augmented Generation (RAG) framework specifically designed for LM-based table understanding. TableRAG leverages query expansion combined with schema and cell retrieval to pinpoint crucial information before providing it to the LMs. This enables more efficient data encoding and precise retrieval, significantly reducing prompt lengths and mitigating information loss. We have developed two new million-token benchmarks from the Arcade and BIRD-SQL datasets to thoroughly evaluate TableRAG's effectiveness at scale. Our results demonstrate that TableRAG's retrieval design achieves the highest retrieval quality, leading to the new state-of-the-art performance on large-scale table understanding.