# Mindful-RAG: A Study of Points of Failure in Retrieval Augmented Generation

arXiv: https://arxiv.org/abs/2407.12216 · HTML: https://mlsanity.com/paper/2407.12216/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Placed in

- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 19 of 19)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [StructGPT: A General Framework for Large Language Model to Reason over Structured Data](https://mlsanity.com/paper/2305.09645/) (2023)
- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [RAFT: Adapting Language Model to Domain Specific RAG](https://mlsanity.com/paper/2403.10131/) (2024)
- [Variational Reasoning for Question Answering with Knowledge Graph](https://mlsanity.com/paper/1709.04071/) (2017)
- [Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering](https://mlsanity.com/paper/2210.02627/) (2022)
- [Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering](https://mlsanity.com/paper/2306.04136/) (2023)
- [Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey](https://mlsanity.com/paper/2311.07914/) (2023)
- [Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases](https://mlsanity.com/paper/2403.10446/) (2024)
- [Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family](https://mlsanity.com/paper/2303.07992/) (2023)
- [Domain-specific Knowledge Graphs: A survey](https://mlsanity.com/paper/2011.00235/) (2020)
- [Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering](https://mlsanity.com/paper/2309.11206/) (2023)
- [HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses](https://mlsanity.com/paper/2312.15883/) (2023)
- [Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge](https://mlsanity.com/paper/2402.12352/) (2024)
- [FIT-RAG: Black-Box RAG with Factual Information and Token Reduction](https://mlsanity.com/paper/2403.14374/) (2024)
- [LB-KBQA: Large-language-model and BERT based Knowledge-Based Question and Answering System](https://mlsanity.com/paper/2402.05130/) (2024)

## Built on by (most cited first, 12 of 12)

- [Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs](https://mlsanity.com/paper/2510.17771/) (2025)
- [HaluMem: Evaluating Hallucinations in Memory Systems of Agents](https://mlsanity.com/paper/2511.03506/) (2025)
- [Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models](https://mlsanity.com/paper/2410.13080/) (2024)
- [Beyond-RAG: Question Identification and Answer Generation in Real-Time Conversations](https://mlsanity.com/paper/2410.10136/) (2024)
- [KOCO-BENCH: Can Large Language Models Leverage Domain Knowledge in Software Development?](https://mlsanity.com/paper/2601.13240/) (2026)
- [RAG-Enabled Intent Reasoning for Application-Network Interaction](https://mlsanity.com/paper/2505.09339/) (2025)
- [Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning](https://mlsanity.com/paper/2505.13994/) (2025)
- [Can we Evaluate RAGs with Synthetic Data?](https://mlsanity.com/paper/2508.11758/) (2025)
- [FinBloom: Knowledge Grounding Large Language Model with Real-time Financial Data](https://mlsanity.com/paper/2502.18471/) (2025)
- [When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs](https://mlsanity.com/paper/2510.09106/) (2025)
- [Investigating Student Interaction Patterns with Large Language Model-Powered Course Assistants in Computer Science Courses](https://mlsanity.com/paper/2509.08862/) (2025)
- [Logic-Oriented Retriever Enhancement via Contrastive Learning](https://mlsanity.com/paper/2602.01116/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are proficient at generating coherent and contextually relevant text but face challenges when addressing knowledge-intensive queries in domain-specific and factual question-answering tasks. Retrieval-augmented generation (RAG) systems mitigate this by incorporating external knowledge sources, such as structured knowledge graphs (KGs). However, LLMs often struggle to produce accurate answers despite access to KG-extracted information containing necessary facts. Our study investigates this dilemma by analyzing error patterns in existing KG-based RAG methods and identifying eight critical failure points. We observed that these errors predominantly occur due to insufficient focus on discerning the question's intent and adequately gathering relevant context from the knowledge graph facts. Drawing on this analysis, we propose the Mindful-RAG approach, a framework designed for intent-based and contextually aligned knowledge retrieval. This method explicitly targets the identified failures and offers improvements in the correctness and relevance of responses provided by LLMs, representing a significant step forward from existing methods.