# Are Finer Citations Always Better? Rethinking Granularity for Attributed Generation

arXiv: https://arxiv.org/abs/2604.01432 · HTML: https://mlsanity.com/paper/2604.01432/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 39 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Builds on (most cited first, 30 of 34)

- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [Efficient Attentions for Long Document Summarization](https://mlsanity.com/paper/2104.02112/) (2021)
- [Large Language Models are Better Reasoners with Self-Verification](https://mlsanity.com/paper/2212.09561/) (2022)
- [Teaching language models to support answers with verified quotes](https://mlsanity.com/paper/2203.11147/) (2022)
- [Long-context LLMs Struggle with Long In-context Learning](https://mlsanity.com/paper/2404.02060/) (2024)
- [Retrieval Head Mechanistically Explains Long-Context Factuality](https://mlsanity.com/paper/2404.15574/) (2024)
- [Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models](https://mlsanity.com/paper/2212.08037/) (2022)
- [LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs](https://mlsanity.com/paper/2406.15319/) (2024)
- [LongAlign: A Recipe for Long Context Alignment of Large Language Models](https://mlsanity.com/paper/2401.18058/) (2024)
- [LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA](https://mlsanity.com/paper/2409.02897/) (2024)
- [Effective Large Language Model Adaptation for Improved Grounding and Citation Generation](https://mlsanity.com/paper/2311.09533/) (2023)
- [Improving Attributed Text Generation of Large Language Models via Preference Learning](https://mlsanity.com/paper/2403.18381/) (2024)
- [Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.00456/) (2024)
- [More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAG](https://mlsanity.com/paper/2503.04388/) (2025)
- [Verifiable Generation with Subsentence-Level Fine-Grained Citations](https://mlsanity.com/paper/2406.06125/) (2024)
- [ALiiCE: Evaluating Positional Fine-grained Citation Generation](https://mlsanity.com/paper/2406.13375/) (2024)
- [Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data](https://mlsanity.com/paper/2404.03862/) (2024)
- [Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-off](https://mlsanity.com/paper/2603.26815/) (2026)
- [CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity](https://mlsanity.com/paper/2404.10513/) (2024)
- [A Survey of Scaling in Large Language Model Reasoning](https://mlsanity.com/paper/2504.02181/) (2025)
- [Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Find](https://mlsanity.com/paper/2505.18148/) (2025)

## Built on by (most cited first, 2 of 2)

- [Attributable by Construction: Claim-Anchored Provenance for Multi-Document Summarization](https://mlsanity.com/paper/2606.23989/) (2026)
- [Tokengeist: Multi-Turn Attribution Tracing in Agentic Conversations](https://mlsanity.com/paper/2607.22610/) (2026)

## Abstract (arXiv)

Citation granularity -- whether to cite individual sentences, paragraphs, or documents -- is a critical design choice in attributed generation. While fine-grained citations are commonly preferred for precise human verification, their impact on model performance remains under-explored. We analyze four model scales (8B-120B) and demonstrate that enforcing fine-grained (sentence-level) citations forfeits gains of 2-97% (median 40%) relative to the best-performing granularity, and up to 338% on individual tasks. Strikingly, setting citation granularity to its optimal value (based on attribution quality) unlocks these substantial gains while leaving overall answer correctness essentially unchanged (between -2.3% and +4.4%). We observe a consistent pattern where attribution quality peaks at intermediate (paragraph-level) granularities: finer citations appear to sever the semantic dependencies needed to ground a claim, while excessively coarse citations introduce distracting noise. Importantly, this performance gap varies with scale: when a claim rests on a small or moderate amount of evidence, it disproportionately penalizes larger models by disrupting the multi-sentence information synthesis at which they excel. Fine-grained citation rests on the premise that a sentence is a sufficient unit of evidence on its own. Our results indicate that it often is not, and that this is a property of the model rather than of the citation standard. Standards fixed for human verifiability may therefore paradoxically degrade the very attribution they aim to ensure; effective attribution requires matching granularity to the model's semantic scope rather than fixing it by convention.