mlsanityData to 7 Oct 2026

ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability

arXiv 2410.11414 ↗First posted 15 Oct 2024cs.CLcited by 100

On the map: Retrieval-augmented generation → Hallucination detection in RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) models are designed to incorporate external knowledge, reducing hallucinations caused by insufficient parametric (internal) knowledge. However, even with accurate and relevant retrieved content, RAG models can still produce hallucinations by generating outputs that conflict with the retrieved information. Detecting such hallucinations requires disentangling how Large Language Models (LLMs) utilize external and parametric knowledge. Current detection methods often focus on one of these mechanisms or without decoupling their intertwined effects, making accurate detection difficult. In this paper, we investigate the internal mechanisms behind hallucinations in RAG scenarios. We discover hallucinations occur when the Knowledge FFNs in LLMs overemphasize parametric knowledge in the residual stream, while Copying Heads fail to effectively retain or integrate external knowledge from retrieved content. Based on these findings, we propose ReDeEP, a novel method that detects hallucinations by decoupling LLM's utilization of external context and parametric knowledge. Our experiments show that ReDeEP significantly improves RAG hallucination detection accuracy. Additionally, we introduce AARF, which mitigates hallucinations by modulating the contributions of Knowledge FFNs and Copying Heads.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 43 of its 58 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

43 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models31 Dec

Other references · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  10. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  12. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
30 more
  1. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  2. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  3. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  4. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  5. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  6. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  7. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  8. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  9. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  10. 2022Confident Adaptive Language Modelingcited by 297
  11. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  12. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  13. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  14. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  15. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  16. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 170
  17. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  18. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  19. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 92
  20. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  21. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 73
  22. 2021Incorporating Residual and Normalization Layers into Analysis of Masked Language Modelscited by 50
  23. 2023Characterizing Mechanisms for Factual Recall in Language Modelscited by 47
  24. 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
  25. 2024SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attentioncited by 39
  26. 2024The Chronicles of RAG: The Retriever, the Chunk and the Generatorcited by 34
  27. 2024UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulationcited by 30
  28. 2024RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Modelscited by 24
  29. 2024Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessmentcited by 19
  30. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queriescited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 51; 2026: 48.

100 papers.

Sort:

By area:

  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  2. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewardingcited by 32
  3. 2025Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancementcited by 24
  4. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  5. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphscited by 18
  6. 2025Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generationcited by 12
  7. 2025Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMscited by 12
  8. 2025ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generationcited by 11
  9. 2025Pre-training Limited Memory Language Models with Internal and External Knowledgecited by 11
  10. 2025LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilizationcited by 11
  11. 2025EigenTrack: Spectral Activation Feature Tracking for Hallucination and Out-of-Distribution Detection in LLMs and VLMscited by 10
  12. 2025From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graphcited by 9
38 more
  1. 2025Uncertainty Quantification in Retrieval Augmented Question Answeringcited by 8
  2. 2025LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signalscited by 8
  3. 2025Investigating CoT Monitorability in Large Reasoning Modelscited by 8
  4. 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigationcited by 7
  5. 2025TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Modelscited by 7
  6. 2025CDF-RAG: Causal Dynamic Feedback for Adaptive Retrieval-Augmented Generationcited by 6
  7. 2025On the Fundamental Impossibility of Hallucination Control in Large Language Modelscited by 6
  8. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  9. 2025FRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Modelscited by 6
  10. 2025Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoderscited by 6
  11. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  12. 2026FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAGcited by 6
  13. 2025UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Modelscited by 5
  14. 2025Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generationcited by 5
  15. 2025Representation Consistency for Accurate and Coherent LLM Answer Aggregationcited by 5
  16. 2025Representation-based Broad Hallucination Detectors Fail to Generalize Out of Distributioncited by 4
  17. 2025LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrievalcited by 4
  18. 2026EvidenceRL: Reinforcing Evidence Consistency for Trustworthy Language Modelscited by 4
  19. 2025SEReDeEP: Hallucination Detection in Retrieval-Augmented Models via Semantic Entropy and Context-Parameter Fusioncited by 3
  20. 2025VeriTrail: Closed-Domain Hallucination Detection with Traceabilitycited by 3
  21. 2025A Survey on AgentOps: Categorization, Challenges, and Future Directionscited by 3
  22. 2026PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifactscited by 3
  23. 2026SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generationcited by 3
  24. 2025Investigating Retrieval-Augmented Generation in Quranic Studies: A Study of 13 Open-Source Large Language Modelscited by 2
  25. 2025An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answeringcited by 2
  26. 2025SR-KI: Scalable and Real-Time Knowledge Integration into LLMs via Supervised Attentioncited by 2
  27. 2025TPA: Next Token Probability Attribution for Detecting Hallucinations in RAGcited by 2
  28. 2026Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agentcited by 2
  29. 2026When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMscited by 2
  30. 2026Small Agent Group is the Future of Digital Healthcited by 2
  31. 2026Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinationscited by 2
  32. 2026What Happens Inside Agent Memory? Circuit Analysis from Emergence to Diagnosiscited by 2
  33. 2026Knowing but Not Correcting: Routine Task Requests Suppress Factual Correction in LLMscited by 2
  34. 2026How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuitscited by 2
  35. 2026Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validationcited by 2
  36. 2026Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decodingcited by 2
  37. 2026FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Modelscited by 2
  38. 2026Visual Token Compression Enhances Robustness of MLLMscited by 2

50 most cited of 100 All 100 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs
  2. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
  3. 2025AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters
  4. 2026Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph
  5. 2024LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation
  6. 2025Theoretical Foundations and Mitigation of Hallucination in Large Language Models
  7. 2026Mitigating Hallucination on Hallucination in RAG via Ensemble Voting
  8. 2024Lynx: An Open Source Hallucination Evaluation Model