PageRecall: Measuring Page Selection in Literature-Grounded Question Answering
Not on the map. Why?
Abstract (arXiv)
We describe our system for LitTraceQA (GroundLM @ EMNLP 2026): given a research question, retrieve the relevant papers from a pool of 27,487, cite the page and the table or figure where the answer lives, and answer in a requested format. Our main finding is that evidence grounding is limited by retrieval, not by reading. The page selector put the annotator's page, which we call the gold page, in front of the model that locates evidence only about half the time (52.6% gold-page recall), while that model, given the page, cited the right one in 45 of the 48 locators it emitted (94%). When the page was missing it rarely said so: of 45 such cases it returned nothing 14 times, a wrong page 24 times, and a correct page 7 times, so the pipeline failed quietly almost twice as often as it failed visibly. Since the failure was that the right page was never shown, the fix is to stop choosing: each retrieved paper fits in the model's context, so we show it whole. Page ranking survives only as a fallback inside papers too long to fit, which no test-split paper was, and gold-page recall reaches 100% on the papers we can parse. Separately, questions that identify their target by position rather than content, such as "the first author of the 24th reference", are served by parsing rather than retrieval: we resolve the bibliography into an addressable list, which also supplies identifiers the evidence metric scores. The final system scores 0.762 paper F1, 0.441 evidence F1 and 0.920 multiple-choice accuracy on the held-out test split. Because the pipeline depends on a closed model without seed control, we release a harness that verifies the paper's central claims against committed artifacts.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 17 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
9 of 17 references.
References · 9
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
- 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
- 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
- 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
- 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 70
- 2026Where Does the Answer Come From? Benchmarking View-Level Visual Evidence Identification in Multi-View MLLMs for Autonomous Drivingcited by 6
- 2026LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answeringcited by 4
Built on by
Not cited yet.