# ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers

arXiv: https://arxiv.org/abs/2607.11074 · HTML: https://mlsanity.com/paper/2607.11074/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 1 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Builds on (most cited first, 1 of 1)

- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)

## Abstract (arXiv)

Large language models are increasingly used to assist scientific reading, but existing evaluation methods often fail to detect whether answers are supported by verifiable citations. We introduce ResearchQA, a benchmark of 6,211 single-paper question-answer pairs from 494 open-access papers spanning eight domains and four question types: lookup, comprehension, multi-hop, and adversarial. ResearchQA is designed for citation-grounded evaluation: it permits multiple valid supporting passages for a claim and rewards grounded refusal when the source paper does not support an answer. We evaluate eight leading closed- and open-weight models in a citation-grounded chat-with-paper setting using a deterministic citation matcher and an LLM-based rubric evaluator. Citation-based metrics separate systems more clearly than LLM-evaluator scores: section coverage and citation accuracy vary substantially across models, while evaluator scores remain tightly compressed. We further find that open-weight models approach the best closed-model citation accuracy while achieving 3 to 6 times lower per-example latency. We release the benchmark, evaluation harness, and evaluator prompt.