# When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation

arXiv: https://arxiv.org/abs/2506.05690 · HTML: https://mlsanity.com/paper/2506.05690/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 56 of its 74 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Placed in

- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 30 of 56)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [LightRAG: Simple and Fast Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.05779/) (2024)
- [HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models](https://mlsanity.com/paper/2405.14831/) (2024)
- [MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries](https://mlsanity.com/paper/2401.15391/) (2024)
- [RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation](https://mlsanity.com/paper/2310.04408/) (2023)
- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [From RAG to Memory: Non-Parametric Continual Learning for Large Language Models](https://mlsanity.com/paper/2502.14802/) (2025)
- [Generation-Augmented Retrieval for Open-domain Question Answering](https://mlsanity.com/paper/2009.08553/) (2020)
- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [Knowledge Graph Prompting for Multi-Document Question Answering](https://mlsanity.com/paper/2308.11730/) (2023)
- [Internet-augmented language models through few-shot prompting for open-domain question answering](https://mlsanity.com/paper/2203.05115/) (2022)
- [Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL](https://mlsanity.com/paper/2406.08426/) (2024)
- [Re2G: Retrieve, Rerank, Generate](https://mlsanity.com/paper/2207.06300/) (2022)
- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [Searching for Best Practices in Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01219/) (2024)
- [MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot](https://mlsanity.com/paper/2502.04413/) (2025)

## Built on by (most cited first, 30 of 48)

- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corpora](https://mlsanity.com/paper/2510.10114/) (2025)
- [LiCoMemory: Lightweight and Cognitive Agentic Memory for Efficient Long-Term Reasoning](https://mlsanity.com/paper/2511.01448/) (2025)
- [You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures](https://mlsanity.com/paper/2508.06105/) (2025)
- [HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking](https://mlsanity.com/paper/2509.11552/) (2025)
- [MemSyco-Bench: Benchmarking Sycophancy in Agent Memory](https://mlsanity.com/paper/2607.01071/) (2026)
- [G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge](https://mlsanity.com/paper/2509.24276/) (2025)
- [AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Construction](https://mlsanity.com/paper/2510.15339/) (2025)
- [AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMs](https://mlsanity.com/paper/2511.05549/) (2025)
- [Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinking](https://mlsanity.com/paper/2508.02435/) (2025)
- [RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing](https://mlsanity.com/paper/2602.00296/) (2026)
- [CSRv2: Unlocking Ultra-Sparse Embeddings](https://mlsanity.com/paper/2602.05735/) (2026)
- [LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning](https://mlsanity.com/paper/2605.28120/) (2026)
- [HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modeling](https://mlsanity.com/paper/2512.23959/) (2025)
- [The Reasoning Bottleneck in Graph-RAG: Structured Prompting and Context Compression for Multi-Hop QA](https://mlsanity.com/paper/2603.14045/) (2026)
- [LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrieval](https://mlsanity.com/paper/2508.07690/) (2025)
- [Beyond RAG for Cyber Threat Intelligence: A Systematic Evaluation of Graph-Based and Agentic Retrieval](https://mlsanity.com/paper/2604.11419/) (2026)
- [Pruning Minimal Reasoning Graphs for Efficient Retrieval-Augmented Generation](https://mlsanity.com/paper/2602.04926/) (2026)
- [LARGER: Lexically Anchored Repository Graph Exploration and Retrieval](https://mlsanity.com/paper/2605.16352/) (2026)
- [When Confidence Takes the Wrong Path: Diagnosing Retrieval-State Lock-In in RAG](https://mlsanity.com/paper/2606.22728/) (2026)
- [MARC: Multimodal and Multi-Task Agentic Retrieval-Augmented Generation for Cold-Start Recommender System](https://mlsanity.com/paper/2511.08181/) (2025)
- [Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment](https://mlsanity.com/paper/2512.09148/) (2025)
- [HELP: HyperNode Expansion and Logical Path-Guided Evidence Localization for Accurate and Efficient GraphRAG](https://mlsanity.com/paper/2602.20926/) (2026)
- [GAM-RAG: Gain-Adaptive Memory for Evolving Retrieval in Retrieval-Augmented Generation](https://mlsanity.com/paper/2603.01783/) (2026)
- [SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges](https://mlsanity.com/paper/2606.15971/) (2026)
- [SkillZip: Contract-Preserving Graph Compression for Scalable Agent Skill Libraries](https://mlsanity.com/paper/2608.05604/) (2026)
- [Coding Agents Aren't Enough! Evaluating an Enterprise Security Brain for Agentic Cloud Investigations](https://mlsanity.com/paper/2609.30345/) (2026)
- [GOSU: Retrieval-Augmented Generation with Global-Level Optimized Semantic Unit-Centric Framework](https://mlsanity.com/paper/2509.00449/) (2025)
- [Evaluating Small Language Models for Agentic On-Farm Decision Support Systems](https://mlsanity.com/paper/2512.14043/) (2025)
- [A Navigational Approach for Comprehensive RAG via Traversal over Proposition Graphs](https://mlsanity.com/paper/2601.04859/) (2026)

## Abstract (arXiv)

Graph retrieval-augmented generation (GraphRAG) has emerged as a powerful paradigm for enhancing large language models (LLMs) with external knowledge. It leverages graphs to model the hierarchical structure between specific concepts, enabling more coherent and effective knowledge retrieval for accurate reasoning.Despite its conceptual promise, recent studies report that GraphRAG frequently underperforms vanilla RAG on many real-world tasks. This raises a critical question: Is GraphRAG really effective, and in which scenarios do graph structures provide measurable benefits for RAG systems? To address this, we propose GraphRAG-Bench, a comprehensive benchmark designed to evaluate GraphRAG models onboth hierarchical knowledge retrieval and deep contextual reasoning. GraphRAG-Bench features a comprehensive dataset with tasks of increasing difficulty, coveringfact retrieval, complex reasoning, contextual summarization, and creative generation, and a systematic evaluation across the entire pipeline, from graph constructionand knowledge retrieval to final generation. Leveraging this novel benchmark, we systematically investigate the conditions when GraphRAG surpasses traditional RAG and the underlying reasons for its success, offering guidelines for its practical application. All related resources and analyses are collected for the community at https://github.com/GraphRAG-Bench/GraphRAG-Benchmark.