# Retrieval-Augmented Generation with Hierarchical Knowledge

arXiv: https://arxiv.org/abs/2503.10150 · HTML: https://mlsanity.com/paper/2503.10150/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Placed in

- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 24 of 24)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [Ragas: Automated Evaluation of Retrieval Augmented Generation](https://mlsanity.com/paper/2309.15217/) (2023)
- [From Louvain to Leiden: guaranteeing well-connected communities](https://mlsanity.com/paper/1810.08473/) (2018)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [LightRAG: Simple and Fast Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.05779/) (2024)
- [MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries](https://mlsanity.com/paper/2401.15391/) (2024)
- [From RAG to Memory: Non-Parametric Continual Learning for Large Language Models](https://mlsanity.com/paper/2502.14802/) (2025)
- [Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs](https://mlsanity.com/paper/2202.05441/) (2022)
- [GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning](https://mlsanity.com/paper/2405.20139/) (2024)
- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [GRAG: Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2405.16506/) (2024)
- [StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization](https://mlsanity.com/paper/2410.08815/) (2024)
- [KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation](https://mlsanity.com/paper/2409.13731/) (2024)
- [Iterative Zero-Shot LLM Prompting for Knowledge Graph Construction](https://mlsanity.com/paper/2307.01128/) (2023)

## Built on by (most cited first, 30 of 47)

- [LeanRAG: Knowledge-Graph-Based Generation with Semantic Aggregation and Hierarchical Retrieval](https://mlsanity.com/paper/2508.10391/) (2025)
- [Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents](https://mlsanity.com/paper/2601.07468/) (2026)
- [HyperMem: Hypergraph Memory for Long-Term Conversations](https://mlsanity.com/paper/2604.08256/) (2026)
- [Cog-RAG: Cognitive-Inspired Dual-Hypergraph with Theme Alignment Retrieval-Augmented Generation](https://mlsanity.com/paper/2511.13201/) (2025)
- [SkillRAE: Agent Skill-Based Context Compilation for Retrieval-Augmented Execution](https://mlsanity.com/paper/2605.10114/) (2026)
- [Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG](https://mlsanity.com/paper/2604.14572/) (2026)
- [MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search](https://mlsanity.com/paper/2604.17265/) (2026)
- [HetaRAG: Hybrid Deep Retrieval-Augmented Generation across Heterogeneous Data Stores](https://mlsanity.com/paper/2509.21336/) (2025)
- [Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Models](https://mlsanity.com/paper/2601.11342/) (2026)
- [An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEs](https://mlsanity.com/paper/2508.21024/) (2025)
- [Embodied Task Planning via Graph-Informed Action Generation with Large Language Models](https://mlsanity.com/paper/2601.21841/) (2026)
- [A/B Agent: A Self-Evolving Agent for Strategy Iteration in Industrial A/B Testing](https://mlsanity.com/paper/2608.04625/) (2026)
- [AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAM](https://mlsanity.com/paper/2510.17934/) (2025)
- [Learning to Debug: LLM-Organized Knowledge Trees for Solving RTL Assertion Failures](https://mlsanity.com/paper/2511.17833/) (2025)
- [TagRAG: Tag-guided Hierarchical Knowledge Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.05254/) (2025)
- [BEAR: Budgeted Evidence Allocation for Multi-Document Reasoning](https://mlsanity.com/paper/2601.18116/) (2026)
- [LLM-Oriented Information Retrieval: A Denoising-First Perspective](https://mlsanity.com/paper/2605.00505/) (2026)
- [ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Models](https://mlsanity.com/paper/2605.10328/) (2026)
- [DeepRefine: Agentic Knowledge Refinement via Reinforcement Learning](https://mlsanity.com/paper/2605.10488/) (2026)
- [DOTRAG: Retrieval-Time Reasoning Along Paths](https://mlsanity.com/paper/2605.18760/) (2026)
- [ContextRAG: Extraction-Free Hierarchical Graph Construction for Retrieval-Augmented Generation](https://mlsanity.com/paper/2605.19735/) (2026)
- [SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges](https://mlsanity.com/paper/2606.15971/) (2026)
- [PonsRAG: A Pons-Inspired RAG Bridging Cognitive Islands for Coordinated Long Narrative Reasoning](https://mlsanity.com/paper/2608.25486/) (2026)
- [GOSU: Retrieval-Augmented Generation with Global-Level Optimized Semantic Unit-Centric Framework](https://mlsanity.com/paper/2509.00449/) (2025)
- [Graph-Guided Concept Selection for Efficient Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.24120/) (2025)
- [PROPEX-RAG: Enhanced GraphRAG using Prompt-Driven Prompt Execution](https://mlsanity.com/paper/2511.01802/) (2025)
- [T-Retriever: Tree-based Hierarchical Retrieval Augmented Generation for Textual Graphs](https://mlsanity.com/paper/2601.04945/) (2026)
- [IGMiRAG: Intuition-Guided Retrieval-Augmented Generation with Adaptive Mining of In-Depth Memory](https://mlsanity.com/paper/2602.07525/) (2026)
- [Hyper-KGGen: A Skill-Driven Knowledge Extractor for High-Quality Knowledge Hypergraph Generation](https://mlsanity.com/paper/2602.19543/) (2026)
- [Adaptive Query Routing: A Tier-Based Framework for Hybrid Retrieval Across Financial, Legal, and Medical Documents](https://mlsanity.com/paper/2604.14222/) (2026)

## Abstract (arXiv)

Graph-based Retrieval-Augmented Generation (RAG) methods have significantly enhanced the performance of large language models (LLMs) in domain-specific tasks. However, existing RAG methods do not adequately utilize the naturally inherent hierarchical knowledge in human cognition, which limits the capabilities of RAG systems. In this paper, we introduce a new RAG approach, called HiRAG, which utilizes hierarchical knowledge to enhance the semantic understanding and structure capturing capabilities of RAG systems in the indexing and retrieval processes. Our extensive experiments demonstrate that HiRAG achieves significant performance improvements over the state-of-the-art baseline methods.