# Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering

arXiv: https://arxiv.org/abs/2403.19631 · HTML: https://mlsanity.com/paper/2403.19631/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering](https://mlsanity.com/paper/2202.13296/) (2022)
- [Model Editing at Scale leads to Gradual and Catastrophic Forgetting](https://mlsanity.com/paper/2401.07453/) (2024)
- [Knowledge Graph Enhanced Large Language Model Editing](https://mlsanity.com/paper/2402.13593/) (2024)
- [Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering](https://mlsanity.com/paper/2005.01218/) (2020)
- [PokeMQA: Programmable knowledge editing for Multi-hop Question Answering](https://mlsanity.com/paper/2312.15194/) (2023)
- [Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering](https://mlsanity.com/paper/1909.07598/) (2019)
- [History Matters: Temporal Knowledge Editing in Large Language Model](https://mlsanity.com/paper/2312.05497/) (2023)
- [Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations](https://mlsanity.com/paper/2306.17256/) (2023)
- [Answering Any-hop Open-domain Questions with Iterative Document Reranking](https://mlsanity.com/paper/2009.07465/) (2020)
- [Simple yet Effective Bridge Reasoning for Open-Domain Multi-Hop Question Answering](https://mlsanity.com/paper/1909.07597/) (2019)

## Built on by (most cited first, 20 of 20)

- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts](https://mlsanity.com/paper/2405.11613/) (2024)
- [Can Knowledge Editing Really Correct Hallucinations?](https://mlsanity.com/paper/2410.16251/) (2024)
- [Knowledge Editing through Chain-of-Thought](https://mlsanity.com/paper/2412.17727/) (2024)
- [MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge](https://mlsanity.com/paper/2412.17032/) (2024)
- [Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness](https://mlsanity.com/paper/2506.05735/) (2025)
- [SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering?](https://mlsanity.com/paper/2502.13233/) (2025)
- [CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners](https://mlsanity.com/paper/2503.16356/) (2025)
- [Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansion](https://mlsanity.com/paper/2502.12598/) (2025)
- [CrashSage: A Large Language Model-Centered Framework for Contextual and Interpretable Traffic Crash Analysis](https://mlsanity.com/paper/2505.07853/) (2025)
- [NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database](https://mlsanity.com/paper/2507.18028/) (2025)
- [Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency](https://mlsanity.com/paper/2505.14309/) (2025)
- [Multi-Hop Reasoning for Question Answering with Hyperbolic Representations](https://mlsanity.com/paper/2507.03612/) (2025)
- [Knowledge Editing for Multi-Hop Question Answering Using Semantic Analysis](https://mlsanity.com/paper/2508.00914/) (2025)
- [Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing](https://mlsanity.com/paper/2511.12661/) (2025)
- [CompactRAG: Reducing LLM Calls and Token Overhead in Multi-Hop Question Answering](https://mlsanity.com/paper/2602.05728/) (2026)
- [MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQA](https://mlsanity.com/paper/2508.07022/) (2025)
- [Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition](https://mlsanity.com/paper/2509.07555/) (2025)
- [Knowledge Homophily in Large Language Models](https://mlsanity.com/paper/2509.23773/) (2025)
- [Making LLMs Truly Forget: Deep Unlearning by Searching, Selecting, and Severing Knowledge Paths](https://mlsanity.com/paper/2609.34442/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown proficiency in question-answering tasks but often struggle to integrate real-time knowledge, leading to potentially outdated or inaccurate responses. This problem becomes even more challenging when dealing with multi-hop questions, since they require LLMs to update and integrate multiple knowledge pieces relevant to the questions. To tackle the problem, we propose the Retrieval-Augmented model Editing (RAE) framework for multi-hop question answering. RAE first retrieves edited facts and then refines the language model through in-context learning. Specifically, our retrieval approach, based on mutual information maximization, leverages the reasoning abilities of LLMs to identify chain facts that traditional similarity-based searches might miss. In addition, our framework includes a pruning strategy to eliminate redundant information from the retrieved facts, which enhances the editing accuracy and mitigates the hallucination problem. Our framework is supported by theoretical justification for its fact retrieval efficacy. Finally, comprehensive evaluation across various LLMs validates RAE's ability in providing accurate answers with updated knowledge. Our code is available at: https://github.com/sycny/RAE.