# CiteEval: Principle-Driven Citation Evaluation for Source Attribution

arXiv: https://arxiv.org/abs/2506.01829 · HTML: https://mlsanity.com/paper/2506.01829/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Builds on (most cited first, 23 of 23)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Teaching language models to support answers with verified quotes](https://mlsanity.com/paper/2203.11147/) (2022)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [Personalization of Large Language Models: A Survey](https://mlsanity.com/paper/2411.00027/) (2024)
- [ExpertQA: Expert-Curated Questions and Attributed Answers](https://mlsanity.com/paper/2309.07852/) (2023)
- [LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA](https://mlsanity.com/paper/2409.02897/) (2024)
- [FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability](https://mlsanity.com/paper/2402.18667/) (2024)
- [MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2501.03468/) (2025)
- [RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering](https://mlsanity.com/paper/2407.13998/) (2024)
- [Learning to Plan and Generate Text with Citations](https://mlsanity.com/paper/2404.03381/) (2024)
- [Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks](https://mlsanity.com/paper/2402.17630/) (2024)
- [Evaluating and Modeling Attribution for Cross-Lingual Question Answering](https://mlsanity.com/paper/2305.14332/) (2023)
- [Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics](https://mlsanity.com/paper/2406.15264/) (2024)

## Built on by (most cited first, 19 of 19)

- [AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery](https://mlsanity.com/paper/2604.05550/) (2026)
- [Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG](https://mlsanity.com/paper/2605.28044/) (2026)
- [Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents](https://mlsanity.com/paper/2605.06635/) (2026)
- [Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents](https://mlsanity.com/paper/2604.03173/) (2026)
- [Epistemic Substitution: How Grokipedia's AI-Generated Encyclopedia Restructures Authority](https://mlsanity.com/paper/2512.03337/) (2025)
- [Architecting Trust in Artificial Epistemic Agents](https://mlsanity.com/paper/2603.02960/) (2026)
- [Exposing Citation Vulnerabilities in Generative Engines](https://mlsanity.com/paper/2510.06823/) (2025)
- [Do LLMs Know They Are Being Tested? Evaluation Awareness and Incentive-Sensitive Failures in GPT-OSS-20B](https://mlsanity.com/paper/2510.08624/) (2025)
- [Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore](https://mlsanity.com/paper/2601.15050/) (2026)
- [Wiki Live Challenge: Challenging Deep Research Agents with Expert-Level Wikipedia Articles](https://mlsanity.com/paper/2602.01590/) (2026)
- [CryptoAnalystBench: Failures in Multi-Tool Long-Form LLM Analysis](https://mlsanity.com/paper/2602.11304/) (2026)
- [Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement](https://mlsanity.com/paper/2603.08924/) (2026)
- [Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact](https://mlsanity.com/paper/2605.14021/) (2026)
- [PubMed-OCR: PMC Open Access OCR Annotations](https://mlsanity.com/paper/2601.11425/) (2026)
- [Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis](https://mlsanity.com/paper/2607.20527/) (2026)
- [Is this Citation on Point?](https://mlsanity.com/paper/2608.12571/) (2026)
- [AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents](https://mlsanity.com/paper/2609.05802/) (2026)
- [CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defense](https://mlsanity.com/paper/2609.15660/) (2026)
- [EvidenT: Building Trustworthy Enterprise Assistants through Evidence Groundedness and Traceability](https://mlsanity.com/paper/2609.22537/) (2026)

## Abstract (arXiv)

Citation quality is crucial in information-seeking systems, directly influencing trust and the effectiveness of information access. Current evaluation frameworks, both human and automatic, mainly rely on Natural Language Inference (NLI) to assess binary or ternary supportiveness from cited sources, which we argue is a suboptimal proxy for citation evaluation. In this work we introduce CiteEval, a citation evaluation framework driven by principles focusing on fine-grained citation assessment within a broad context, encompassing not only the cited sources but the full retrieval context, user query, and generated text. Guided by the proposed framework, we construct CiteBench, a multi-domain benchmark with high-quality human annotations on citation quality. To enable efficient evaluation, we further develop CiteEval-Auto, a suite of model-based metrics that exhibit strong correlation with human judgments. Experiments across diverse systems demonstrate CiteEval-Auto's superior ability to capture the multifaceted nature of citations compared to existing metrics, offering a principled and scalable approach to evaluate and improve model-generated citations.