mlsanityData to 7 Oct 2026

Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey

arXiv 2311.07914 ↗First posted 14 Nov 2023cs.CL cs.LGcited by 92

Not on the map. Why?

Abstract (arXiv)

The contemporary LLMs are prone to producing hallucinations, stemming mainly from the knowledge gaps within the models. To address this critical limitation, researchers employ diverse strategies to augment the LLMs by incorporating external knowledge, aiming to reduce hallucinations and enhance reasoning accuracy. Among these strategies, leveraging knowledge graphs as a source of external information has demonstrated promising results. In this survey, we comprehensively review these knowledge-graph-based augmentation techniques in LLMs, focusing on their efficacy in mitigating hallucinations. We systematically categorize these methods into three overarching groups, offering methodological comparisons and performance evaluations. Lastly, this survey explores the current trends and challenges associated with these techniques and outlines potential avenues for future research in this emerging field.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Builds on 87 of its 127 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

87 of 127 references.

References · 87

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2019Language Models as Knowledge Bases?cited by 1,922
  12. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
38 more
  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  2. 2019How Can We Know What Language Models Know?cited by 953
  3. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  4. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  5. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  6. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  7. 2022Large Language Models Can Self-Improvecited by 656
  8. 2020Knowledge Graphscited by 619
  9. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  10. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  11. 2023Augmented Language Models: a Surveycited by 398
  12. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  13. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  14. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  15. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  16. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
  17. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
  18. 2023How Language Model Hallucinations Can Snowballcited by 310
  19. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
  20. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  21. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  22. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  23. 2022Binding Language Models in Symbolic Languagescited by 224
  24. 2023Large Language Model Guided Tree-of-Thoughtcited by 218
  25. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  26. 2023OpenAGI: When LLM Meets Domain Expertscited by 196
  27. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  28. 2020A Survey on Knowledge Graph-Based Recommender Systemscited by 170
  29. 2022A Review on Language Models as Knowledge Basescited by 156
  30. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  31. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  32. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  33. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  34. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  35. 2019E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTcited by 96
  36. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativenesscited by 76
  37. 2019Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modelingcited by 71
  38. 2023Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledgecited by 66

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 35; 2025: 31; 2026: 23.

92 papers.

Sort:

By area:

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  2. 2024Evaluating Retrieval Quality in Retrieval-Augmented Generationcited by 90
  3. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  4. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMscited by 48
  5. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  6. 2024FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agentscited by 39
  7. 2024A Survey on Large Language Model Hallucination via a Creativity Perspectivecited by 38
  8. 2024FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answeringcited by 38
  9. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  10. 2024GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Frameworkcited by 30
  11. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  12. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
38 more
  1. 2024T-RAG: Lessons from the LLM Trenchescited by 23
  2. 2024Materials science in the era of large language models: a perspectivecited by 23
  3. 2024Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Modelscited by 23
  4. 2024Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Modelscited by 20
  5. 2024MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Modelscited by 20
  6. 2024Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answeringcited by 20
  7. 2024Dual Reasoning: A GNN-LLM Collaborative Framework for Knowledge Graph Question Answeringcited by 19
  8. 2024A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterizationcited by 18
  9. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  10. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 17
  11. 2024Logical Consistency of Large Language Models in Fact-checkingcited by 15
  12. 2024Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximizationcited by 14
  13. 2023Large Knowledge Model: Perspectives and Challengescited by 12
  14. 2024Mindful-RAG: A Study of Points of Failure in Retrieval Augmented Generationcited by 12
  15. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  16. 2024Evaluating the Retrieval Component in LLM-Based Question Answering Systemscited by 11
  17. 2024Thought Graph: Generating Thought Process for Biological Reasoningcited by 10
  18. 2024Beyond-RAG: Question Identification and Answer Generation in Real-Time Conversationscited by 9
  19. 2024LLM vs. Lawyers: Identifying a Subset of Summary Judgments in a Large UK Case Law Datasetcited by 8
  20. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoningcited by 8
  21. 2024CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoningcited by 7
  22. 2024Evaluating Consistency and Reasoning Capabilities of Large Language Modelscited by 7
  23. 2024Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memescited by 7
  24. 2025Evaluating the Effect of Retrieval Augmentation on Social Biasescited by 7
  25. 2024Consolidating Trees of Robotic Plans Generated Using Large Language Models to Improve Reliabilitycited by 6
  26. 2024LinkQ: An LLM-Assisted Visual Interface for Knowledge Graph Question-Answeringcited by 6
  27. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  28. 2024Harnessing the Power of Semi-Structured Knowledge and LLMs with Triplet-Based Prefiltering for Question Answeringcited by 4
  29. 2025A Lightweight Approach to Detection of AI-Generated Texts Using Stylometric Featurescited by 4
  30. 2024DepsRAG: Towards Agentic Reasoning and Planning for Software Dependency Managementcited by 3
  31. 2025From Documents to Dialogue: Building KG-RAG Enhanced AI Assistantscited by 3
  32. 2025RedditESS: A Mental Health Social Support Interaction Dataset -- Understanding Effective Social Support to Refine AI-Driven Support Toolscited by 3
  33. 2025Lie to Me: Knowledge Graphs for Robust Hallucination Self-Detection in LLMscited by 3
  34. 2026Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agentscited by 3
  35. 2024Enhancing Supply Chain Visibility with Generative AI: An Exploratory Case Study on Relationship Prediction in Knowledge Graphscited by 2
  36. 2025Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Modelscited by 2
  37. 2025Boosting Knowledge Graph-based Recommendations through Confidence-Aware Augmentation with Large Language Modelscited by 2
  38. 2025LLM-Powered Knowledge Graphs for Enterprise Intelligence and Analyticscited by 2

50 most cited of 92 All 92 (JSON)