mlsanityData to 14 Oct 2026

RAGged Edges: The Double-Edged Sword of Retrieval-Augmented Chatbots

arXiv 2403.01193 ↗First posted 2 Mar 2024cs.CL cs.AIcited by 11

On the map: Retrieval-augmented generation → Hallucination detection in RAG

Abstract (arXiv)

Large language models (LLMs) like ChatGPT demonstrate the remarkable progress of artificial intelligence. However, their tendency to hallucinate -- generate plausible but false information -- poses a significant challenge. This issue is critical, as seen in recent court cases where ChatGPT's use led to citations of non-existent legal rulings. This paper explores how Retrieval-Augmented Generation (RAG) can counter hallucinations by integrating external knowledge with prompts. We empirically evaluate RAG against standard LLMs using prompts designed to induce hallucinations. Our results show that RAG increases accuracy in some cases, but can still be misled when prompts directly contradict the model's pre-trained understanding. These findings highlight the complex nature of hallucinations and the need for more robust solutions to ensure LLM reliability in real-world applications. We offer practical recommendations for RAG deployment and discuss implications for the development of more trustworthy LLMs.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 10 of its 20 references that are in the map. First posted on arXiv 2 Mar 2024.

Builds on

10 of 20 references.

References · 10

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  6. 2023Large Language Modelscited by 780
  7. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  8. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 169
  9. 2023A Categorical Archive of ChatGPT Failurescited by 168
  10. 2023Trapping LLM Hallucinations Using Tagged Context Promptscited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 4; 2026: 1.

11 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 113
  3. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  4. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  5. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 30
  6. 2025Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Datacited by 10
  7. 2024Understanding Retrieval Robustness for Retrieval-Augmented Image Captioningcited by 9
  8. 2024Assessing "Implicit" Retrieval Robustness of Large Language Modelscited by 9
  9. 2024Optimizing and Evaluating Enterprise Retrieval-Augmented Generation (RAG): A Content Design Perspectivecited by 7
  10. 2025LLM-based Corroborating and Refuting Evidence Retrieval for Scientific Claim Verificationcited by 5
  11. 2026PAVE: Premise-Aware Validation and Editing for Retrieval-Augmented LLMscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024100% Elimination of Hallucinations on RAGTruth for GPT-4 and GPT-3.5 Turbo
  2. 2025Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards
  3. 2024Citation-Enhanced Generation for LLM-based Chatbots
  4. 2026Mitigating Hallucination on Hallucination in RAG via Ensemble Voting
  5. 2025Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systems
  6. 2025Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning
  7. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
  8. 2025Enhancing Critical Thinking with AI: A Tailored Warning System for RAG Models