# Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing

arXiv: https://arxiv.org/abs/2410.06331 · HTML: https://mlsanity.com/paper/2410.06331/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 44 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [Analyzing Transformers in Embedding Space](https://mlsanity.com/paper/2209.02535/) (2022)
- [Language Models Implement Simple Word2Vec-style Vector Arithmetic](https://mlsanity.com/paper/2305.16130/) (2023)
- [Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries](https://mlsanity.com/paper/2406.12775/) (2024)
- [Massive Editing for Large Language Models via Meta Learning](https://mlsanity.com/paper/2311.04661/) (2023)
- [A Unified Framework for Model Editing](https://mlsanity.com/paper/2403.14236/) (2024)
- [Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.14491/) (2023)
- [Understanding and Patching Compositional Reasoning in LLMs](https://mlsanity.com/paper/2402.14328/) (2024)
- [WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing](https://mlsanity.com/paper/2402.10987/) (2024)
- [Multi-hop Question Answering under Temporal Knowledge Editing](https://mlsanity.com/paper/2404.00492/) (2024)
- [A Survey on Knowledge Editing of Neural Networks](https://mlsanity.com/paper/2310.19704/) (2023)

## Built on by (most cited first, 30 of 37)

- [When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models](https://mlsanity.com/paper/2508.02087/) (2025)
- [Understanding the Repeat Curse in Large Language Models from a Feature Perspective](https://mlsanity.com/paper/2504.14218/) (2025)
- [EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification](https://mlsanity.com/paper/2502.06852/) (2025)
- [Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs](https://mlsanity.com/paper/2506.07180/) (2025)
- [When Modalities Conflict: How Unimodal Reasoning Uncertainty Governs Preference Dynamics in MLLMs](https://mlsanity.com/paper/2511.02243/) (2025)
- [How do Transformers Learn Implicit Reasoning?](https://mlsanity.com/paper/2505.23653/) (2025)
- [Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models](https://mlsanity.com/paper/2508.10599/) (2025)
- [Towards User-level Private Reinforcement Learning with Human Feedback](https://mlsanity.com/paper/2502.17515/) (2025)
- [Understanding How Value Neurons Shape the Generation of Specified Values in LLMs](https://mlsanity.com/paper/2505.17712/) (2025)
- [CODEMENV: Benchmarking Large Language Models on Code Migration](https://mlsanity.com/paper/2506.00894/) (2025)
- [Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning](https://mlsanity.com/paper/2502.09022/) (2025)
- [PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibration](https://mlsanity.com/paper/2510.10205/) (2025)
- [CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners](https://mlsanity.com/paper/2503.16356/) (2025)
- [Predicting LLM Output Length via Entropy-Guided Representations](https://mlsanity.com/paper/2602.11812/) (2026)
- [DocMEdit: Towards Document-Level Model Editing](https://mlsanity.com/paper/2505.19572/) (2025)
- [COMPKE: Complex Question Answering under Knowledge Editing](https://mlsanity.com/paper/2506.00829/) (2025)
- [Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models](https://mlsanity.com/paper/2508.02360/) (2025)
- [Controlling Repetition in Protein Language Models](https://mlsanity.com/paper/2602.00782/) (2026)
- [Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localization](https://mlsanity.com/paper/2509.07864/) (2025)
- [Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models](https://mlsanity.com/paper/2512.17911/) (2025)
- [The Compositional Architecture of Regret in Large Language Models](https://mlsanity.com/paper/2506.15617/) (2025)
- [ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall](https://mlsanity.com/paper/2510.07896/) (2025)
- [Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models](https://mlsanity.com/paper/2601.03542/) (2026)
- [Understanding the Dynamics of Demonstration Conflict in In-Context Learning](https://mlsanity.com/paper/2603.04464/) (2026)
- [Word Recovery in Large Language Models Enables Character-Level Tokenization Robustness](https://mlsanity.com/paper/2603.10771/) (2026)
- [Matryoshka Concept Bottleneck Models](https://mlsanity.com/paper/2605.20612/) (2026)
- [Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images](https://mlsanity.com/paper/2506.07184/) (2025)
- [EtCon: Edit-then-Consolidate for Reliable Knowledge Editing](https://mlsanity.com/paper/2512.04753/) (2025)
- [Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Models](https://mlsanity.com/paper/2602.00300/) (2026)
- [Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping](https://mlsanity.com/paper/2603.06923/) (2026)

## Abstract (arXiv)

The locate-then-edit paradigm has shown significant promise for knowledge editing (KE) in Large Language Models (LLMs). While previous methods perform well on single-hop fact recall tasks, they consistently struggle with multi-hop factual recall tasks involving newly edited knowledge. In this paper, leveraging tools in mechanistic interpretability, we first identify that in multi-hop tasks, LLMs tend to retrieve knowledge with implicit subject information from deeper MLP layers, unlike single-hop tasks, which rely on shallow layers. This distinction explains the poor performance of current methods in multi-hop queries, as they primarily focus on editing shallow layers with single-hop edit prompts, leaving deeper layers unchanged. To address this, we propose IFMET, a novel locate-then-edit KE approach designed to edit both shallow and deep MLP layers. Beyond single-hop editing prompts, IFMET further incorporates multi-hop editing prompts to locate and modify knowledge across different stages of reasoning. Experimental results demonstrate that IFMET significantly improves performance on multi-hop factual recall tasks, overcoming the limitations of previous locate-then-edit methods