# LettuceDetect: A Hallucination Detection Framework for RAG Applications

arXiv: https://arxiv.org/abs/2502.17125 · HTML: https://mlsanity.com/paper/2502.17125/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Placed in

- Retrieval-augmented generation / [Hallucination detection in RAG](https://mlsanity.com/rag/hallucination-detection-in-rag/)

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools](https://mlsanity.com/paper/2405.20362/) (2024)
- [LM vs LM: Detecting Factual Errors via Cross Examination](https://mlsanity.com/paper/2305.13281/) (2023)
- [Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literature](https://mlsanity.com/paper/2310.16146/) (2023)
- [FACTOID: FACtual enTailment fOr hallucInation Detection](https://mlsanity.com/paper/2403.19113/) (2024)
- [Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics](https://mlsanity.com/paper/2403.08904/) (2024)

## Built on by (most cited first, 30 of 34)

- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA](https://mlsanity.com/paper/2510.04849/) (2025)
- [HALT: Hallucination Assessment via Log-probs as Time series](https://mlsanity.com/paper/2602.02888/) (2026)
- [CLATTER: Comprehensive Entailment Reasoning for Hallucination Detection](https://mlsanity.com/paper/2506.05243/) (2025)
- [FRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Models](https://mlsanity.com/paper/2507.20930/) (2025)
- [FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG](https://mlsanity.com/paper/2601.05866/) (2026)
- [Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future](https://mlsanity.com/paper/2604.27924/) (2026)
- [Confundo: Learning to Generate Robust Poison for Practical RAG Systems](https://mlsanity.com/paper/2602.06616/) (2026)
- [Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents](https://mlsanity.com/paper/2607.00895/) (2026)
- [Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategies](https://mlsanity.com/paper/2503.07306/) (2025)
- [Turk-LettuceDetect: A Hallucination Detection Models for Turkish RAG Applications](https://mlsanity.com/paper/2509.17671/) (2025)
- [ORION Grounded in Context: Retrieval-Based Method for Hallucination Detection](https://mlsanity.com/paper/2504.15771/) (2025)
- [Listen to the Context: Towards Faithful Large Language Models for Retrieval Augmented Generation on Climate Questions](https://mlsanity.com/paper/2505.15633/) (2025)
- [InterpDetect: Interpretable Signals for Detecting Hallucinations in Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.21538/) (2025)
- [A Geometric Taxonomy of Hallucinations in LLMs](https://mlsanity.com/paper/2602.13224/) (2026)
- [Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAG](https://mlsanity.com/paper/2603.27752/) (2026)
- [RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration](https://mlsanity.com/paper/2604.15945/) (2026)
- [Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)](https://mlsanity.com/paper/2607.04223/) (2026)
- [Temporal Multi-Signal Fusion for Token-Level Hallucination Detection](https://mlsanity.com/paper/2608.18115/) (2026)
- [HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domain](https://mlsanity.com/paper/2509.11619/) (2025)
- [Semantic Grounding Index: Geometric Bounds on Context Engagement in RAG Systems](https://mlsanity.com/paper/2512.13771/) (2025)
- [Fast and Faithful: Real-Time Verification for Long-Document Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2603.23508/) (2026)
- [A multilingual hallucination benchmark: MultiWikiQHalluA](https://mlsanity.com/paper/2605.02504/) (2026)
- [Do Benchmarks Underestimate LLM Performance? Evaluating Hallucination Detection With LLM-First Human-Adjudicated Assessment](https://mlsanity.com/paper/2605.08462/) (2026)
- [Detecting Hallucinations for Large Language Model-based Knowledge Graph Reasoning](https://mlsanity.com/paper/2606.19351/) (2026)
- [SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems](https://mlsanity.com/paper/2608.00033/) (2026)
- [Hallucination Span Detection with Input-Side Evidence Alignment](https://mlsanity.com/paper/2608.15804/) (2026)
- [Detecting and Repairing Hallucinations in Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.29307/) (2026)
- [Enoki: Efficient Multi-Level Hallucination Detection](https://mlsanity.com/paper/2609.00581/) (2026)
- [Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection](https://mlsanity.com/paper/2609.05025/) (2026)

## Abstract (arXiv)

Retrieval Augmented Generation (RAG) systems remain vulnerable to hallucinated answers despite incorporating external knowledge sources. We present LettuceDetect a framework that addresses two critical limitations in existing hallucination detection methods: (1) the context window constraints of traditional encoder-based methods, and (2) the computational inefficiency of LLM based approaches. Building on ModernBERT's extended context capabilities (up to 8k tokens) and trained on the RAGTruth benchmark dataset, our approach outperforms all previous encoder-based models and most prompt-based models, while being approximately 30 times smaller than the best models. LettuceDetect is a token-classification model that processes context-question-answer triples, allowing for the identification of unsupported claims at the token level. Evaluations on the RAGTruth corpus demonstrate an F1 score of 79.22% for example-level detection, which is a 14.8% improvement over Luna, the previous state-of-the-art encoder-based architecture. Additionally, the system can process 30 to 60 examples per second on a single GPU, making it more practical for real-world RAG applications.