Are Finer Citations Always Better? Rethinking Granularity for Attributed Generation
Not on the map. Why?
Abstract (arXiv)
Citation granularity -- whether to cite individual sentences, paragraphs, or documents -- is a critical design choice in attributed generation. While fine-grained citations are commonly preferred for precise human verification, their impact on model performance remains under-explored. We analyze four model scales (8B-120B) and demonstrate that enforcing fine-grained (sentence-level) citations forfeits gains of 2-97% (median 40%) relative to the best-performing granularity, and up to 338% on individual tasks. Strikingly, setting citation granularity to its optimal value (based on attribution quality) unlocks these substantial gains while leaving overall answer correctness essentially unchanged (between -2.3% and +4.4%). We observe a consistent pattern where attribution quality peaks at intermediate (paragraph-level) granularities: finer citations appear to sever the semantic dependencies needed to ground a claim, while excessively coarse citations introduce distracting noise. Importantly, this performance gap varies with scale: when a claim rests on a small or moderate amount of evidence, it disproportionately penalizes larger models by disrupting the multi-sentence information synthesis at which they excel. Fine-grained citation rests on the premise that a sentence is a sufficient unit of evidence on its own. Our results indicate that it often is not, and that this is a property of the model rather than of the citation standard. Standards fixed for human verifiability may therefore paradoxically degrade the very attribution they aim to ensure; effective attribution requires matching granularity to the model's semantic scope rather than fixing it by convention.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 39 references that are in the map. First posted on arXiv 1 Apr 2026.
Builds on
34 of 39 references.
References · 34
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2021Efficient Attentions for Long Document Summarizationcited by 342
- 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
22 more
- 2022Teaching language models to support answers with verified quotescited by 290
- 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
- 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
- 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
- 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
- 2024LongAlign: A Recipe for Long Context Alignment of Large Language Modelscited by 102
- 2024LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QAcited by 78
- 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
- 2024Improving Attributed Text Generation of Large Language Models via Preference Learningcited by 23
- 2024Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generationcited by 23
- 2025More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAGcited by 16
- 2024Verifiable Generation with Subsentence-Level Fine-Grained Citationscited by 15
- 2024ALiiCE: Evaluating Positional Fine-grained Citation Generationcited by 15
- 2024Verifiable by Design: Aligning Language Models to Quote from Pre-Training Datacited by 13
- 2026Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-offcited by 13
- 2024CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularitycited by 12
- 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
- 2025Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Findcited by 7
- 2025Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Modelscited by 7
- 2025Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Modelscited by 3
- 2025Citation-Grounded Code Comprehension: Preventing LLM Hallucination Through Hybrid Retrieval and Graph-Augmented Contextcited by 2
- 2025Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generationcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.