# TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance

arXiv: https://arxiv.org/abs/2105.07624 · HTML: https://mlsanity.com/paper/2105.07624/

## Key facts

- Cited by 408 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text](https://mlsanity.com/paper/1904.09537/) (2019)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [Open Question Answering over Tables and Text](https://mlsanity.com/paper/2010.10439/) (2020)
- [Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure](https://mlsanity.com/paper/2004.05080/) (2020)
- [NumNet: Machine Reading Comprehension with Numerical Reasoning](https://mlsanity.com/paper/1910.06701/) (2019)
- [Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension](https://mlsanity.com/paper/1909.00109/) (2019)
- [Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering](https://mlsanity.com/paper/1607.06275/) (2016)
- [Question Directed Graph Attention Network for Numerical Reasoning over Text](https://mlsanity.com/paper/2009.07448/) (2020)
- [Summarizing and Exploring Tabular Data in Conversational Search](https://mlsanity.com/paper/2005.11490/) (2020)
- [Explicit State Tracking with Semi-Supervision for Neural Dialogue Generation](https://mlsanity.com/paper/1808.10596/) (2018)
- [Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks](https://mlsanity.com/paper/1704.08384/) (2017)
- [Auto-completion for Data Cells in Relational Tables](https://mlsanity.com/paper/1909.03443/) (2019)
- [A deep learning approach for predicting the quality of online health expert question-answering services](https://mlsanity.com/paper/1612.07040/) (2016)

## Built on by (most cited first, 30 of 408)

- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024)
- [Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning](https://mlsanity.com/paper/2209.14610/) (2022)
- [What matters when building vision-language models?](https://mlsanity.com/paper/2405.02246/) (2024)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [Making Large Language Models Better Reasoners with Step-Aware Verifier](https://mlsanity.com/paper/2206.02336/) (2022)
- [RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs](https://mlsanity.com/paper/2407.02485/) (2024)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [TheoremQA: A Theorem-driven Question Answering dataset](https://mlsanity.com/paper/2305.12524/) (2023)
- [MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://mlsanity.com/paper/2509.18154/) (2025)
- [Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://mlsanity.com/paper/2309.15402/) (2023)
- [Building and better understanding vision-language models: insights and future directions](https://mlsanity.com/paper/2408.12637/) (2024)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [Active Prompting with Chain-of-Thought for Large Language Models](https://mlsanity.com/paper/2302.12246/) (2023)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [TableBench: A Comprehensive and Complex Benchmark for Table Question Answering](https://mlsanity.com/paper/2408.09174/) (2024)
- [MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale](https://mlsanity.com/paper/2412.05237/) (2024)
- [A Survey of Deep Learning for Mathematical Reasoning](https://mlsanity.com/paper/2212.10535/) (2022)
- [MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data](https://mlsanity.com/paper/2206.01347/) (2022)
- [HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation](https://mlsanity.com/paper/2108.06712/) (2021)
- [RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2407.11005/) (2024)
- [Large Language Models are few(1)-shot Table Reasoners](https://mlsanity.com/paper/2210.06710/) (2022)
- [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023)
- [SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation](https://mlsanity.com/paper/2406.14991/) (2024)
- [The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models](https://mlsanity.com/paper/2406.05761/) (2024)
- [HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction](https://mlsanity.com/paper/2408.04948/) (2024)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [ChatQA: Surpassing GPT-4 on Conversational QA and RAG](https://mlsanity.com/paper/2401.10225/) (2024)
- [ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding](https://mlsanity.com/paper/2501.05452/) (2025)
- [Towards Complex Document Understanding By Discrete Reasoning](https://mlsanity.com/paper/2207.11871/) (2022)

## Abstract (arXiv)

Hybrid data combining both tabular and textual content (e.g., financial reports) are quite pervasive in the real world. However, Question Answering (QA) over such hybrid data is largely neglected in existing research. In this work, we extract samples from real financial reports to build a new large-scale QA dataset containing both Tabular And Textual data, named TAT-QA, where numerical reasoning is usually required to infer the answer, such as addition, subtraction, multiplication, division, counting, comparison/sorting, and the compositions. We further propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text. It adopts sequence tagging to extract relevant cells from the table along with relevant spans from the text to infer their semantics, and then applies symbolic reasoning over them with a set of aggregation operators to arrive at the final answer. TAGOPachieves 58.0% inF1, which is an 11.1% absolute increase over the previous best baseline model, according to our experiments on TAT-QA. But this result still lags far behind performance of expert human, i.e.90.8% in F1. It is demonstrated that our TAT-QA is very challenging and can serve as a benchmark for training and testing powerful QA models that address hybrid form data.