mlsanityData to 14 Oct 2026

LettuceDetect: A Hallucination Detection Framework for RAG Applications

arXiv 2502.17125 ↗First posted 24 Feb 2025cs.CL cs.AIcited by 34

On the map: Retrieval-augmented generation → Hallucination detection in RAG

Abstract (arXiv)

Retrieval Augmented Generation (RAG) systems remain vulnerable to hallucinated answers despite incorporating external knowledge sources. We present LettuceDetect a framework that addresses two critical limitations in existing hallucination detection methods: (1) the context window constraints of traditional encoder-based methods, and (2) the computational inefficiency of LLM based approaches. Building on ModernBERT's extended context capabilities (up to 8k tokens) and trained on the RAGTruth benchmark dataset, our approach outperforms all previous encoder-based models and most prompt-based models, while being approximately 30 times smaller than the best models. LettuceDetect is a token-classification model that processes context-question-answer triples, allowing for the identification of unsupported claims at the token level. Evaluations on the RAGTruth corpus demonstrate an F1 score of 79.22% for example-level detection, which is a 14.8% improvement over Luna, the previous state-of-the-art encoder-based architecture. Additionally, the system can process 30 to 60 examples per second on a single GPU, making it more practical for real-world RAG applications.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 20 of its 29 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

20 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics13 Mar
  2. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models31 Dec

Other references · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  9. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  10. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  11. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  12. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 265
6 more
  1. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 265
  2. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  3. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 176
  4. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
  5. 2023Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literaturecited by 29
  6. 2024FACTOID: FACtual enTailment fOr hallucInation Detectioncited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 23.

34 papers.

Sort:

By area:

  1. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
  2. 2025When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQAcited by 14
  3. 2026HALT: Hallucination Assessment via Log-probs as Time seriescited by 8
  4. 2025CLATTER: Comprehensive Entailment Reasoning for Hallucination Detectioncited by 7
  5. 2025FRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Modelscited by 6
  6. 2026FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAGcited by 6
  7. 2026Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Futurecited by 6
  8. 2026Confundo: Learning to Generate Robust Poison for Practical RAG Systemscited by 5
  9. 2026Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documentscited by 3
  10. 2025Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategiescited by 2
  11. 2025Turk-LettuceDetect: A Hallucination Detection Models for Turkish RAG Applicationscited by 2
  12. 2025ORION Grounded in Context: Retrieval-Based Method for Hallucination Detectioncited by 1
22 more
  1. 2025Listen to the Context: Towards Faithful Large Language Models for Retrieval Augmented Generation on Climate Questionscited by 1
  2. 2025InterpDetect: Interpretable Signals for Detecting Hallucinations in Retrieval-Augmented Generationcited by 1
  3. 2026A Geometric Taxonomy of Hallucinations in LLMscited by 1
  4. 2026Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAGcited by 1
  5. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integrationcited by 1
  6. 2026Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)cited by 1
  7. 2026Temporal Multi-Signal Fusion for Token-Level Hallucination Detectioncited by 1
  8. 2025HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domaincited by 0
  9. 2025Semantic Grounding Index: Geometric Bounds on Context Engagement in RAG Systemscited by 0
  10. 2026Fast and Faithful: Real-Time Verification for Long-Document Retrieval-Augmented Generation Systemscited by 0
  11. 2026A multilingual hallucination benchmark: MultiWikiQHalluAcited by 0
  12. 2026Do Benchmarks Underestimate LLM Performance? Evaluating Hallucination Detection With LLM-First Human-Adjudicated Assessmentcited by 0
  13. 2026Detecting Hallucinations for Large Language Model-based Knowledge Graph Reasoningcited by 0
  14. 2026SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systemscited by 0
  15. 2026Hallucination Span Detection with Input-Side Evidence Alignmentcited by 0
  16. 2026Detecting and Repairing Hallucinations in Retrieval-Augmented Generationcited by 0
  17. 2026Enoki: Efficient Multi-Level Hallucination Detectioncited by 0
  18. 2026Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detectioncited by 0
  19. 2026AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documentscited by 0
  20. 2026RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detectioncited by 0
  21. 2026Grounding Probes: Generator-Independent Hallucination Detection from Observer Model Hidden Statescited by 0
  22. 2026ToolHACE: A Dataset for Detecting Hallucinations in Tool-Based LLM Responsescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Osiris: A Lightweight Open-Source Hallucination Detection System
  2. 2025MetaRAG: Metamorphic Testing for Hallucination Detection in RAG Systems
  3. 2025HalluMix: A Task-Agnostic, Multi-Domain Benchmark for Real-World Hallucination Detection
  4. 2025TPA: Next Token Probability Attribution for Detecting Hallucinations in RAG
  5. 2025LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals
  6. 2025Probabilistic distances-based hallucination detection in LLMs with RAG
  7. 2025Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Rate
  8. 2025Bi'an: A Bilingual Benchmark and Model for Hallucination Detection in Retrieval-Augmented Generation