# Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations

arXiv: https://arxiv.org/abs/2405.02228 · HTML: https://mlsanity.com/paper/2405.02228/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 3 May 2024.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 12 of 12)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Structural Scaffolds for Citation Intent Classification in Scientific Publications](https://mlsanity.com/paper/1904.01608/) (2019)
- [Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models](https://mlsanity.com/paper/2212.08037/) (2022)
- [Source-Aware Training Enables Knowledge Attribution in Language Models](https://mlsanity.com/paper/2404.01019/) (2024)
- [RefSum: Refactoring Neural Summarization](https://mlsanity.com/paper/2104.07210/) (2021)

## Built on by (most cited first, 10 of 10)

- [NarraBench: A Comprehensive Framework for Narrative Benchmarking](https://mlsanity.com/paper/2510.09869/) (2025)
- [Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attribution](https://mlsanity.com/paper/2509.21557/) (2025)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions](https://mlsanity.com/paper/2509.02452/) (2025)
- [Neurosymbolic AI approach to Attribution in Large Language Models](https://mlsanity.com/paper/2410.03726/) (2024)
- [The Attribution Crisis in LLM Search Results](https://mlsanity.com/paper/2508.00838/) (2025)
- [Aligning Large Language Model Behavior with Human Citation Preferences](https://mlsanity.com/paper/2602.05205/) (2026)
- [IoT-Based Preventive Mental Health Using Knowledge Graphs and Standards for Better Well-Being](https://mlsanity.com/paper/2406.13791/) (2024)
- [NeuroLit Navigator: A Neurosymbolic Approach to Scholarly Article Searches for Systematic Reviews](https://mlsanity.com/paper/2503.00278/) (2025)
- [Experiments or Outcomes? Probing Scientific Feasibility in Large Language Models](https://mlsanity.com/paper/2604.18786/) (2026)

## Abstract (arXiv)

Large language models (LLMs) increasingly generate citation-backed responses, yet citation hallucination remains a major challenge for trustworthy scientific information access. We introduce REASONS, a benchmark of 12,723 sentence-level citation instances spanning 12 arXiv subject categories, designed to evaluate scientific citation attribution under varying evidence conditions. We propose a dual-metric framework consisting of Abstention Rate (AR) and Hallucination Rate (HR) to characterize the trade-off between reliability and responsiveness. Using author-attribution and title-attribution tasks, we evaluate proprietary and open-source LLMs under zero-context, metadata-augmented, cascaded metadata-augmented prompting (CMP), retrieval-augmented, and adversarial settings. Advanced RAG lowers HR relative to Naive RAG (65.4% vs. 87.6%) but reduces AR from 5.0% to 0%. Under adversarial metadata, several systems exceed 85% HR, while retrieval-augmented variants frequently maintain near-zero abstention. Human evaluation of 1,000 outputs (kappa=0.78) finds a 12.7:1 ratio of factual hallucinations to acceptable paraphrases. Our findings demonstrate that citation attribution systems should be evaluated not only for correctness but also for their ability to abstain appropriately under uncertainty. REASONS provides a benchmark and evaluation framework for studying attribution reliability in citation generation.