# Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting

arXiv: https://arxiv.org/abs/2311.13314 · HTML: https://mlsanity.com/paper/2311.13314/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Large-scale Simple Question Answering with Memory Networks](https://mlsanity.com/paper/1506.02075/) (2015)
- [WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences](https://mlsanity.com/paper/2306.07906/) (2023)
- [Improved Natural Language Generation via Loss Truncation](https://mlsanity.com/paper/2004.14589/) (2020)
- [Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering](https://mlsanity.com/paper/2210.01613/) (2022)
- [Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation](https://mlsanity.com/paper/1910.08684/) (2019)
- [PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions](https://mlsanity.com/paper/2305.14908/) (2023)
- [RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding](https://mlsanity.com/paper/2212.01588/) (2022)
- [Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View](https://mlsanity.com/paper/2102.08585/) (2021)

## Built on by (most cited first, 30 of 70)

- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides](https://mlsanity.com/paper/2501.03936/) (2025)
- [StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization](https://mlsanity.com/paper/2410.08815/) (2024)
- [On Limitations of the Transformer Architecture](https://mlsanity.com/paper/2402.08164/) (2024)
- [Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey](https://mlsanity.com/paper/2402.05391/) (2024)
- [Visual Agents as Fast and Slow Thinkers](https://mlsanity.com/paper/2408.08862/) (2024)
- [Large Language Models Hallucination: A Comprehensive Survey](https://mlsanity.com/paper/2510.06265/) (2025)
- [Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning](https://mlsanity.com/paper/2410.14211/) (2024)
- [A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoning](https://mlsanity.com/paper/2501.02497/) (2025)
- [DeepRAG: Thinking to Retrieve Step by Step for Large Language Models](https://mlsanity.com/paper/2502.01142/) (2025)
- [A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions](https://mlsanity.com/paper/2406.03712/) (2024)
- [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2404.14928/) (2024)
- [Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning](https://mlsanity.com/paper/2502.02871/) (2025)
- [KnowGPT: Knowledge Graph based Prompting for Large Language Models](https://mlsanity.com/paper/2312.06185/) (2023)
- [Exploring Autonomous Agents through the Lens of Large Language Models: A Review](https://mlsanity.com/paper/2404.04442/) (2024)
- [Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities](https://mlsanity.com/paper/2505.20099/) (2025)
- [Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective](https://mlsanity.com/paper/2411.14258/) (2024)
- [DMQR-RAG: Diverse Multi-Query Rewriting for RAG](https://mlsanity.com/paper/2411.13154/) (2024)
- [Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Survey](https://mlsanity.com/paper/2408.12880/) (2024)
- [Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory](https://mlsanity.com/paper/2405.16674/) (2024)
- [Mitigating Hallucination in Fictional Character Role-Play](https://mlsanity.com/paper/2406.17260/) (2024)
- [MKGL: Mastery of a Three-Word Language](https://mlsanity.com/paper/2410.07526/) (2024)
- [Mitigating Hallucinations in Large Language Models via Self-Refinement-Enhanced Knowledge Retrieval](https://mlsanity.com/paper/2405.06545/) (2024)
- [Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering](https://mlsanity.com/paper/2411.11504/) (2024)
- [EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs](https://mlsanity.com/paper/2406.01238/) (2024)
- [Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answering](https://mlsanity.com/paper/2403.01390/) (2024)
- [SRAG: Structured Retrieval-Augmented Generation for Multi-Entity Question Answering over Wikipedia Graph](https://mlsanity.com/paper/2503.01346/) (2025)
- [1+1>2: Can Large Language Models Serve as Cross-Lingual Knowledge Aggregators?](https://mlsanity.com/paper/2406.14721/) (2024)
- [StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluation](https://mlsanity.com/paper/2408.03281/) (2024)
- [Knowledge Verification to Nip Hallucination in the Bud](https://mlsanity.com/paper/2401.10768/) (2024)

## Abstract (arXiv)

Incorporating factual knowledge in knowledge graph is regarded as a promising approach for mitigating the hallucination of large language models (LLMs). Existing methods usually only use the user's input to query the knowledge graph, thus failing to address the factual hallucination generated by LLMs during its reasoning process. To address this problem, this paper proposes Knowledge Graph-based Retrofitting (KGR), a new framework that incorporates LLMs with KGs to mitigate factual hallucination during the reasoning process by retrofitting the initial draft responses of LLMs based on the factual knowledge stored in KGs. Specifically, KGR leverages LLMs to extract, select, validate, and retrofit factual statements within the model-generated responses, which enables an autonomous knowledge verifying and refining procedure without any additional manual efforts. Experiments show that KGR can significantly improve the performance of LLMs on factual QA benchmarks especially when involving complex reasoning processes, which demonstrates the necessity and effectiveness of KGR in mitigating hallucination and enhancing the reliability of LLMs.