# SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types

arXiv: https://arxiv.org/abs/2412.11757 · HTML: https://mlsanity.com/paper/2412.11757/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 25 of 25)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [emrQA: A Large Corpus for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1809.00732/) (2018)
- [TableBench: A Comprehensive and Complex Benchmark for Table Question Answering](https://mlsanity.com/paper/2408.09174/) (2024)
- [Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing](https://mlsanity.com/paper/2404.12253/) (2024)
- [SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research](https://mlsanity.com/paper/2308.13149/) (2023)
- [The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4](https://mlsanity.com/paper/2311.07361/) (2023)
- [CriticBench: Benchmarking LLMs for Critique-Correct Reasoning](https://mlsanity.com/paper/2402.14809/) (2024)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables](https://mlsanity.com/paper/2305.13186/) (2023)
- [DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documents](https://mlsanity.com/paper/2311.09805/) (2023)
- [FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domains](https://mlsanity.com/paper/2311.09797/) (2023)
- [SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models](https://mlsanity.com/paper/2401.07950/) (2024)
- [BIOMRC: A Dataset for Biomedical Machine Reading Comprehension](https://mlsanity.com/paper/2005.06376/) (2020)
- [BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra](https://mlsanity.com/paper/2402.17882/) (2024)
- [Enhancing Financial Question Answering with a Multi-Agent Reflection Framework](https://mlsanity.com/paper/2410.21741/) (2024)
- [Exploring Hybrid Question Answering via Program-based Prompting](https://mlsanity.com/paper/2402.10812/) (2024)

## Built on by (most cited first, 10 of 10)

- [Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation](https://mlsanity.com/paper/2510.09671/) (2025)
- [LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering](https://mlsanity.com/paper/2608.07370/) (2026)
- [MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering](https://mlsanity.com/paper/2502.17253/) (2025)
- [Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoning](https://mlsanity.com/paper/2506.05278/) (2025)
- [HCT-QA: A Benchmark for Question Answering on Human-Centric Tables](https://mlsanity.com/paper/2504.20047/) (2025)
- [When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors](https://mlsanity.com/paper/2606.32029/) (2026)
- [An Actionable Diagnosis of Multilingual, Multi-Agent Planning Failures](https://mlsanity.com/paper/2608.03735/) (2026)
- [ModelTables: A Corpus of Tables about Models](https://mlsanity.com/paper/2512.16106/) (2025)
- [SciTaRC: A Plan-Annotated Scientific Tabular QA Benchmark for Language Reasoning and Complex Computation](https://mlsanity.com/paper/2603.08910/) (2026)
- [Latent Bridges for Multi-Table Question Answering](https://mlsanity.com/paper/2606.28916/) (2026)

## Abstract (arXiv)

Scientific question answering (SQA) is an important task aimed at answering questions based on papers. However, current SQA datasets have limited reasoning types and neglect the relevance between tables and text, creating a significant gap with real scenarios. To address these challenges, we propose a QA benchmark for scientific tables and text with diverse reasoning types (SciTaT). To cover more reasoning types, we summarize various reasoning types from real-world questions. To involve both tables and text, we require the questions to incorporate tables and text as much as possible. Based on SciTaT, we propose a strong baseline (CaR), which combines various reasoning methods to address different reasoning types and process tables and text at the same time. CaR brings average improvements of 12.9% over other baselines on SciTaT, validating its effectiveness. Error analysis reveals the challenges of SciTaT, such as complex numerical calculations and domain knowledge.