mlsanityData to 7 Oct 2026

Unifying Large Language Models and Knowledge Graphs: A Roadmap

arXiv 2306.08302 ↗First posted 14 Jun 2023cs.CL cs.AIcited by 663

Not on the map. Why?

Cited by 87 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs), such as ChatGPT and GPT4, are making new waves in the field of natural language processing and artificial intelligence, due to their emergent ability and generalizability. However, LLMs are black-box models, which often fall short of capturing and accessing factual knowledge. In contrast, Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge. KGs can enhance LLMs by providing external knowledge for inference and interpretability. Meanwhile, KGs are difficult to construct and evolving by nature, which challenges the existing methods in KGs to generate new facts and represent unseen knowledge. Therefore, it is complementary to unify LLMs and KGs together and simultaneously leverage their advantages. In this article, we present a forward-looking roadmap for the unification of LLMs and KGs. Our roadmap consists of three general frameworks, namely, 1) KG-enhanced LLMs, which incorporate KGs during the pre-training and inference phases of LLMs, or for the purpose of enhancing understanding of the knowledge learned by LLMs; 2) LLM-augmented KGs, that leverage LLMs for different KG tasks such as embedding, completion, construction, graph-to-text generation, and question answering; and 3) Synergized LLMs + KGs, in which LLMs and KGs play equal roles and work in a mutually beneficial way to enhance both LLMs and KGs for bidirectional reasoning driven by both data and knowledge. We review and summarize existing efforts within these three frameworks in our roadmap and pinpoint their future research directions.

Key facts. Cited by 663 papers in the map (data through 7 Oct 2026). Builds on 193 of its 300 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

193 of 300 references.

References · 193

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  11. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
38 more
  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2020Longformer: The Long-Document Transformercited by 3,362
  3. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2023A Survey of Large Language Modelscited by 2,537
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2018Deep contextualized word representationscited by 2,285
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  11. 2019Language Models as Knowledge Bases?cited by 1,922
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  13. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  14. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  15. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  16. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  17. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  18. 2022Mass-Editing Memory in a Transformercited by 976
  19. 2019How Can We Know What Language Models Know?cited by 953
  20. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
  21. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  22. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  23. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  24. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  25. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  26. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  27. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  28. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  29. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  30. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  31. 2021Fast Model Editing at Scalecited by 491
  32. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  33. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  34. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  35. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
  36. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  37. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  38. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339

50 most cited of 193 All 193 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 78; 2024: 229; 2025: 177; 2026: 179.

663 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  3. 2024Large Language Models: A Surveycited by 506
  4. 2024Understanding the planning of LLM agents: A surveycited by 414
  5. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  6. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
  7. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  8. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  9. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 235
  10. 2023Language is All a Graph Needscited by 219
  11. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  12. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
38 more
  1. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  2. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  3. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  4. 2024GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoningcited by 128
  5. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  6. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 100
  7. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  8. 2024ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoningcited by 95
  9. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  10. 2024KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graphcited by 90
  11. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 89
  12. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 89
  13. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  14. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  15. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  16. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 78
  17. 2024NetLLM: Adapting Large Language Models for Networkingcited by 76
  18. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73
  19. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  20. 2024A Survey of Large Language Models for Graphscited by 71
  21. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  22. 2025HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representationcited by 68
  23. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  24. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  25. 2023Graph Neural Prompting with Large Language Modelscited by 64
  26. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  27. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  28. 2024Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answeringcited by 59
  29. 2024KG-RAG: Bridging the Gap Between Knowledge and Creativitycited by 55
  30. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  31. 2024DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literaturecited by 50
  32. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  33. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  34. 2026MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agentscited by 49
  35. 2023Making Large Language Models Perform Better in Knowledge Graph Completioncited by 48
  36. 2024Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Modelscited by 47
  37. 2024Prompt Design and Engineering: Introduction and Advanced Methodscited by 45
  38. 2023ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Modelscited by 44

50 most cited of 663 All 663 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling
  2. 2024Synergizing Knowledge Graphs with Large Language Models: A Comprehensive Review and Future Prospects ↗
  3. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  4. 2025Enhancing Large Language Models with Reliable Knowledge Graphs
  5. 2025KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs
  6. 2023KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models
  7. 2023Knowledge Graphs are not Created Equal: Exploring the Properties and Structure of Real KGs
  8. 2023Enhancing Knowledge Graph Construction Using Large Language Models