# Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards

arXiv: https://arxiv.org/abs/2505.04847 · HTML: https://mlsanity.com/paper/2505.04847/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 7 May 2025.

## Placed in

- Retrieval-augmented generation / [Hallucination detection in RAG](https://mlsanity.com/rag/hallucination-detection-in-rag/)

## Builds on (most cited first, 30 of 35)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [AlignScore: Evaluating Factual Consistency with a Unified Alignment Function](https://mlsanity.com/paper/2305.16739/) (2023)
- [Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](https://mlsanity.com/paper/2104.13346/) (2021)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization](https://mlsanity.com/paper/2112.08542/) (2021)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [Detecting Hallucinated Content in Conditional Neural Sequence Generation](https://mlsanity.com/paper/2011.02593/) (2020)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [Human-like Summarization Evaluation with ChatGPT](https://mlsanity.com/paper/2304.02554/) (2023)
- [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://mlsanity.com/paper/2205.12854/) (2022)
- [FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"](https://mlsanity.com/paper/2410.03727/) (2024)
- [MeetingBank: A Benchmark Dataset for Meeting Summarization](https://mlsanity.com/paper/2305.17529/) (2023)
- [TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models](https://mlsanity.com/paper/2305.11171/) (2023)

## Built on by (most cited first, 20 of 20)

- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [Deep Research: A Survey of Autonomous Research Agents](https://mlsanity.com/paper/2508.12752/) (2025)
- [Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders](https://mlsanity.com/paper/2512.08892/) (2025)
- [MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination](https://mlsanity.com/paper/2603.24579/) (2026)
- [Structured Prompts Improve Evaluation of Language Models](https://mlsanity.com/paper/2511.20836/) (2025)
- [PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality](https://mlsanity.com/paper/2606.20897/) (2026)
- [Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey](https://mlsanity.com/paper/2601.03262/) (2025)
- [PrivacyAlign: Contextual Privacy Alignment for LLM Agents](https://mlsanity.com/paper/2606.21710/) (2026)
- [GSAR: Typed Grounding for Hallucination Detection and Recovery in Multi-Agent LLMs](https://mlsanity.com/paper/2604.23366/) (2026)
- [LLM-Based Intelligent Notification Composition: From Static Personalization to Context-Aware Persuasive Messaging](https://mlsanity.com/paper/2605.16264/) (2026)
- [Summarization is Not Dead Yet](https://mlsanity.com/paper/2606.08000/) (2026)
- [BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation](https://mlsanity.com/paper/2606.15893/) (2026)
- [Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI](https://mlsanity.com/paper/2607.17883/) (2026)
- [A corrective agentic hybrid RAG and an operations-grounded evaluation for a scientific facility](https://mlsanity.com/paper/2607.24663/) (2026)
- [GuidedRAG: Semantic Steering of Retrieval-Augmented Generation](https://mlsanity.com/paper/2607.26071/) (2026)
- [SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems](https://mlsanity.com/paper/2608.00033/) (2026)
- [Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.30996/) (2026)
- [AquiLLM: Evaluating Faithfulness in Open-Weight RAG-LLM Systems for Scientific Research](https://mlsanity.com/paper/2609.16519/) (2026)
- [RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection](https://mlsanity.com/paper/2609.39229/) (2026)
- [Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Models](https://mlsanity.com/paper/2610.05472/) (2026)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) aims to reduce hallucinations by grounding responses in external context, yet large language models (LLMs) still frequently introduce unsupported information or contradictions even when provided with relevant context. This paper presents two complementary efforts at Vectara to measure and benchmark LLM faithfulness in RAG. First, we describe our original hallucination leaderboard, which has tracked hallucination rates for LLMs since 2023 using our HHEM hallucination detection model. Motivated by limitations observed in current hallucination detection methods, we introduce FaithJudge, an LLM-as-a-judge framework that leverages a pool of diverse human-annotated hallucination examples to substantially improve the automated hallucination evaluation of LLMs. We introduce an enhanced hallucination leaderboard centered on FaithJudge that benchmarks LLMs on RAG faithfulness in summarization, question-answering, and data-to-text generation tasks. FaithJudge enables a more reliable benchmarking of LLM hallucinations in RAG and supports the development of more trustworthy generative AI systems: https://github.com/vectara/FaithJudge.