Knowledge Pyramid Construction for Multi-Level Retrieval-Augmented Generation
On the map: Retrieval-augmented generation → Knowledge-graph and structured-knowledge RAG
Abstract (arXiv)
This paper addresses the need for improved precision in existing knowledge-enhanced question-answering frameworks, specifically Retrieval-Augmented Generation (RAG) methods that primarily focus on enhancing recall. We propose a multi-layer knowledge pyramid approach within the RAG framework to achieve a better balance between precision and recall. The knowledge pyramid consists of three layers: Ontologies, Knowledge Graphs (KGs), and chunk-based raw text. We employ cross-layer augmentation techniques for comprehensive knowledge coverage and dynamic updates of the Ontology schema and instances. To ensure compactness, we utilize cross-layer filtering methods for knowledge condensation in KGs. Our approach, named PolyRAG, follows a waterfall model for retrieval, starting from the top of the pyramid and progressing down until a confident answer is obtained. We introduce two benchmarks for domain-specific knowledge retrieval, one in the academic domain and the other in the financial domain. The effectiveness of the methods has been validated through comprehensive experiments by outperforming 19 SOTA methods. An encouraging observation is that the proposed method has augmented the GPT-4, providing 395% F1 gain by improving its performance from 0.1636 to 0.8109.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation. Builds on 35 of its 56 references that are in the map. First posted on arXiv 31 Jul 2024.
Builds on
35 of 56 references.
References · 35
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2023A Survey on Evaluation of Large Language Modelscited by 1,227
- 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
23 more
- 2023BloombergGPT: A Large Language Model for Financecited by 923
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
- 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
- 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 211
- 2023OpenAGI: When LLM Meets Domain Expertscited by 196
- 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
- 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
- 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
- 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 88
- 2020SCATTER: Selective Context Attentional Scene Text Recognizercited by 69
- 2023FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasetscited by 68
- 2023Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answeringcited by 64
- 2023DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuningcited by 64
- 2023Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Textcited by 48
- 2024From human experts to machines: An LLM supported approach to ontology and knowledge graph constructioncited by 40
- 2024Knowledge Graphs as Context Sources for LLM-Based Explanations of Learning Recommendationscited by 31
- 2023Iterative Zero-Shot LLM Prompting for Knowledge Graph Constructioncited by 25
- 2023Fine-tuning Large Enterprise Language Models via Ontological Reasoningcited by 14
- 2019Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognitioncited by 11
- 2023OntoChatGPT Information System: Ontology-Driven Structured Prompts for ChatGPT Meta-Learningcited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generation
- 2024KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation
- 2025KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering
- 2025DSRAG: A Domain-Specific Retrieval Framework Based on Document-derived Multimodal Knowledge Graph
- 2025MMKB-RAG: A Multi-Modal Knowledge-Based Retrieval-Augmented Generation Framework
- 2024Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization
- 2025Context-Guided Dynamic Retrieval for Improving Generation Quality in RAG Models
- 2024DuetRAG: Collaborative Retrieval-Augmented Generation