# KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases

arXiv: https://arxiv.org/abs/2308.11761 · HTML: https://mlsanity.com/paper/2308.11761/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 17 Aug 2023.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings](https://mlsanity.com/paper/2305.11554/) (2023)
- [Internet-Augmented Dialogue Generation](https://mlsanity.com/paper/2107.07566/) (2021)
- [PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text](https://mlsanity.com/paper/1904.09537/) (2019)
- [ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory](https://mlsanity.com/paper/2306.03901/) (2023)
- [Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering](https://mlsanity.com/paper/2306.04136/) (2023)
- [Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision](https://mlsanity.com/paper/1611.00020/) (2016)
- [UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph](https://mlsanity.com/paper/2212.00959/) (2022)
- [Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering](https://mlsanity.com/paper/2202.13296/) (2022)
- [Document-Level Machine Translation with Large Language Models](https://mlsanity.com/paper/2304.02210/) (2023)
- [DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases](https://mlsanity.com/paper/2210.00063/) (2022)
- [Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT](https://mlsanity.com/paper/2304.11116/) (2023)
- [RET-LLM: Towards a General Read-Write Memory for Large Language Models](https://mlsanity.com/paper/2305.14322/) (2023)
- [TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph](https://mlsanity.com/paper/2104.07302/) (2021)
- [Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Models](https://mlsanity.com/paper/2303.05977/) (2023)
- [Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments](https://mlsanity.com/paper/2212.09736/) (2022)
- [SPARQA: Skeleton-based Semantic Parsing for Complex Questions over Knowledge Bases](https://mlsanity.com/paper/2003.13956/) (2020)
- [Program Transfer for Answering Complex Questions over Knowledge Bases](https://mlsanity.com/paper/2110.05743/) (2021)
- [Crake: Causal-Enhanced Table-Filler for Question Answering over Large Scale Knowledge Base](https://mlsanity.com/paper/2207.03680/) (2022)

## Built on by (most cited first, 30 of 52)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023)
- [Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.20724/) (2024)
- [A Survey on Retrieval-Augmented Text Generation for Large Language Models](https://mlsanity.com/paper/2404.10981/) (2024)
- [Demystifying Chains, Trees, and Graphs of Thoughts](https://mlsanity.com/paper/2401.14295/) (2024)
- [Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models](https://mlsanity.com/paper/2407.06089/) (2024)
- [What is the Role of Small Models in the LLM Era: A Survey](https://mlsanity.com/paper/2409.06857/) (2024)
- [M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions](https://mlsanity.com/paper/2405.16420/) (2024)
- [RAR-b: Reasoning as Retrieval Benchmark](https://mlsanity.com/paper/2404.06347/) (2024)
- [KnowPhish: Large Language Models Meet Multimodal Knowledge Graphs for Enhancing Reference-Based Phishing Detection](https://mlsanity.com/paper/2403.02253/) (2024)
- [AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering](https://mlsanity.com/paper/2510.05445/) (2025)
- [GraphRAG under Fire](https://mlsanity.com/paper/2501.14050/) (2025)
- [FlowAgent: Achieving Compliance and Flexibility for Workflow Agents](https://mlsanity.com/paper/2502.14345/) (2025)
- [PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generation](https://mlsanity.com/paper/2501.11551/) (2025)
- [Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey](https://mlsanity.com/paper/2504.10499/) (2025)
- [Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression](https://mlsanity.com/paper/2505.19433/) (2025)
- [EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answering](https://mlsanity.com/paper/2604.05149/) (2026)
- [Aligning Large Language Models to a Domain-specific Graph Database for NL2GQL](https://mlsanity.com/paper/2402.16567/) (2024)
- [Efficient Knowledge Infusion via KG-LLM Alignment](https://mlsanity.com/paper/2406.03746/) (2024)
- [RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering](https://mlsanity.com/paper/2410.22353/) (2024)
- [DH-RAG: A Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue](https://mlsanity.com/paper/2502.13847/) (2025)
- [NG-Router: Graph-Supervised Multi-Agent Collaboration for Nutrition Question Answering](https://mlsanity.com/paper/2510.09854/) (2025)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2404.17897/) (2024)
- [DFA-RAG: Conversational Semantic Router for Large Language Model with Definite Finite Automaton](https://mlsanity.com/paper/2402.04411/) (2024)
- [DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation](https://mlsanity.com/paper/2506.01954/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive impact in the field of natural language processing, but they still struggle with several issues regarding, such as completeness, timeliness, faithfulness and adaptability. While recent efforts have focuses on connecting LLMs with external knowledge sources, the integration of knowledge bases (KBs) remains understudied and faces several challenges. In this paper, we introduce KnowledGPT, a comprehensive framework to bridge LLMs with various knowledge bases, facilitating both the retrieval and storage of knowledge. The retrieval process employs the program of thought prompting, which generates search language for KBs in code format with pre-defined functions for KB operations. Besides retrieval, KnowledGPT offers the capability to store knowledge in a personalized KB, catering to individual user demands. With extensive experiments, we show that by integrating LLMs with KBs, KnowledGPT properly answers a broader range of questions requiring world knowledge compared with vanilla LLMs, utilizing both knowledge existing in widely-known KBs and extracted into personalized KBs.