mlsanityData to 7 Oct 2026

SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types

arXiv 2412.11757 ↗First posted 16 Dec 2024cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Scientific question answering (SQA) is an important task aimed at answering questions based on papers. However, current SQA datasets have limited reasoning types and neglect the relevance between tables and text, creating a significant gap with real scenarios. To address these challenges, we propose a QA benchmark for scientific tables and text with diverse reasoning types (SciTaT). To cover more reasoning types, we summarize various reasoning types from real-world questions. To involve both tables and text, we require the questions to incorporate tables and text as much as possible. Based on SciTaT, we propose a strong baseline (CaR), which combines various reasoning methods to address different reasoning types and process tables and text at the same time. CaR brings average improvements of 12.9% over other baselines on SciTaT, validating its effectiveness. Error analysis reveals the challenges of SciTaT, such as complex numerical calculations and domain knowledge.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 16 Dec 2024.

Builds on

25 of 42 references.

References · 25

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  2. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  3. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  4. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  5. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  6. 2022Galactica: A Large Language Model for Sciencecited by 736
  7. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  8. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  9. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  10. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  11. 2018emrQA: A Large Corpus for Question Answering on Electronic Medical Recordscited by 134
  12. 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 127
13 more
  1. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  2. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 112
  3. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  4. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoningcited by 88
  5. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  6. 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tablescited by 51
  7. 2023DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documentscited by 45
  8. 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 39
  9. 2024SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Modelscited by 38
  10. 2020BIOMRC: A Dataset for Biomedical Machine Reading Comprehensioncited by 35
  11. 2024BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebracited by 23
  12. 2024Enhancing Financial Question Answering with a Multi-Agent Reflection Frameworkcited by 12
  13. 2024Exploring Hybrid Question Answering via Program-based Promptingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluationcited by 4
  2. 2026LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answeringcited by 4
  3. 2025MULTITAT: Benchmarking Multilingual Table-and-Text Question Answeringcited by 2
  4. 2025HCT-QA: A Benchmark for Question Answering on Human-Centric Tablescited by 1
  5. 2025Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoningcited by 1
  6. 2026When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errorscited by 1
  7. 2025ModelTables: A Corpus of Tables about Modelscited by 0
  8. 2026SciTaRC: A Plan-Annotated Scientific Tabular QA Benchmark for Language Reasoning and Complex Computationcited by 0
  9. 2026Latent Bridges for Multi-Table Question Answeringcited by 0
  10. 2026An Actionable Diagnosis of Multilingual, Multi-Agent Planning Failurescited by 0