MT-RAIG: Novel Benchmark and Evaluation Framework for Retrieval-Augmented Insight Generation over Multiple Tables
On the map: Retrieval-augmented generation → Knowledge-graph and structured-knowledge RAG
Abstract (arXiv)
Recent advancements in table-based reasoning have expanded beyond factoid-level QA to address insight-level tasks, where systems should synthesize implicit knowledge in the table to provide explainable analyses. Although effective, existing studies remain confined to scenarios where a single gold table is given alongside the user query, failing to address cases where users seek comprehensive insights from multiple unknown tables. To bridge these gaps, we propose MT-RAIG Bench, design to evaluate systems on Retrieval-Augmented Insight Generation over Mulitple-Tables. Additionally, to tackle the suboptimality of existing automatic evaluation methods in the table domain, we further introduce a fine-grained evaluation framework MT-RAIG Eval, which achieves better alignment with human quality judgments on the generated insights. We conduct extensive experiments and reveal that even frontier LLMs still struggle with complex multi-table reasoning, establishing our MT-RAIG Bench as a challenging testbed for future research.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 62 references that are in the map. First posted on arXiv 17 Feb 2025.
Builds on
11 of 62 references.
References · 11
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024GPT-4o System Cardcited by 4,467
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 201
- 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 136
- 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
- 2024Evaluating Language Models as Synthetic Data Generatorscited by 32
- 2024Synthetic Data in AI: Challenges, Applications, and Ethical Implicationscited by 21
- 2023Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?cited by 16
- 2024Stop Playing the Guessing Game! Target-free User Simulation for Evaluating Conversational Recommender Systemscited by 4
Built on by
Citing papers per year: 2025: 5; 2026: 2.
7 papers.
- 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
- 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 17
- 2025TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoningcited by 5
- 2025Can Large Language Models be Effective Online Opinion Miners?cited by 3
- 2025IPQA: A Benchmark for Core Intent Identification in Personalized Question Answeringcited by 2
- 2026Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexitiescited by 1
- 2026ExpVoyager: Direct Experience Navigation for Dynamic Agent Skill Synthesiscited by 0
- 2026ExpVoyager: Direct Experience Navigation for Dynamic Agent Skill Synthesis26 Sep 2026
- 2026Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexities7 Jul 2026
- 2025IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering27 Oct 2025
- 2025TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning7 Oct 2025
- 2025Deep Research: A Survey of Autonomous Research Agents18 Aug 2025
- 2025Can Large Language Models be Effective Online Opinion Miners?21 May 2025
- 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking2 Apr 2025
Similar, not cited
Close in wording, no citation either way
- 2026FT-RAG: A Fine-grained Retrieval-Augmented Generation Framework for Complex Table Reasoning
- 2025Structure-Augmented Reasoning Generation
- 2026PARTAB: Partition-Aware Reasoning with Structured Evidence for Scalable Table Understanding
- 2025TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models
- 2026When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models
- 2024Knowledge in Triples for LLMs: Enhancing Table QA Accuracy with Semantic Extraction
- 2026Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore
- 2026QUIETT: Query-Independent Table Transformation for Robust Reasoning