# LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering

arXiv: https://arxiv.org/abs/2608.07370 · HTML: https://mlsanity.com/paper/2608.07370/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation.
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Placed in

- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)

## Builds on (most cited first, 13 of 13)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [PaperQA: Retrieval-Augmented Generative Agent for Scientific Research](https://mlsanity.com/paper/2312.07559/) (2023)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubrics](https://mlsanity.com/paper/2509.00496/) (2025)
- [Evidence Inference 2.0: More Data, Better Models](https://mlsanity.com/paper/2005.04177/) (2020)
- [M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2411.04075/) (2024)
- [SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types](https://mlsanity.com/paper/2412.11757/) (2024)
- [Where Does the Answer Come From? Benchmarking View-Level Visual Evidence Identification in Multi-View MLLMs for Autonomous Driving](https://mlsanity.com/paper/2606.09644/) (2026)
- [AirQA: A Comprehensive QA Dataset for AI Research with Instance-Level Evaluation](https://mlsanity.com/paper/2509.16952/) (2025)

## Built on by (most cited first, 4 of 4)

- [Lit3R: Retrieve-Relate-Read for Evidence-Grounded Question Answering over Scientific Literature](https://mlsanity.com/paper/2609.16912/) (2026)
- [PageRecall: Measuring Page Selection in Literature-Grounded Question Answering](https://mlsanity.com/paper/2609.18154/) (2026)
- [TRACE: Target-Aware Retrieval, Attributed Evidence, and Contract-Constrained Extraction for LitTraceQA](https://mlsanity.com/paper/2609.38861/) (2026)
- [GroundSight at GroundLM 2026 Shared Tasks: GoldenViewVQA](https://mlsanity.com/paper/2610.11402/) (2026)

## Abstract (arXiv)

Scientific literature is increasingly used as a knowledge source for language models, retrieval-augmented generation systems, and research assistants, but answering research questions from papers requires more than fluent generation. A reliable system must identify the relevant papers, locate the concrete evidence that supports the answer, and produce a response that is faithful to that evidence. We present LitTraceQA, a benchmark for literature-grounded question answering over scientific papers. Given a research question and a metadata pool of papers, a system must return three connected outputs: canonical paper identifiers, supporting evidence locations, and answers in one or more requested formats, including free-form text, multiple-choice answers, and structured tables. LitTraceQA targets evidence types common in scientific reading: tables, figures, text spans, equations or algorithms, and citation contexts. The public development split contains 55 examples, including 26 hidden-source single-paper questions and 29 multi-paper questions, and provides gold papers, evidence annotations, and answers for local validation. We also analyze a larger final annotation collection with 4,978 unique-question records over 4,859 unique gold papers. By evaluating paper retrieval, evidence grounding, and answer accuracy separately, LitTraceQA provides a testbed for scientific QA systems that produce verifiable answers rather than unsupported summaries.