# InstructEdit: Instruction-based Knowledge Editing for Large Language Models

arXiv: https://arxiv.org/abs/2402.16123 · HTML: https://mlsanity.com/paper/2402.16123/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 63 references that are in the map.
- First posted on arXiv 25 Feb 2024.

## Builds on (most cited first, 30 of 54)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models](https://mlsanity.com/paper/2301.04213/) (2023)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [Linearity of Relation Decoding in Transformer Language Models](https://mlsanity.com/paper/2308.09124/) (2023)
- [Transformer-Patcher: One Mistake worth One Neuron](https://mlsanity.com/paper/2301.09785/) (2023)
- [Knowledge Editing for Large Language Models: A Survey](https://mlsanity.com/paper/2310.16218/) (2023)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue](https://mlsanity.com/paper/2401.04700/) (2024)
- [EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models](https://mlsanity.com/paper/2308.07269/) (2023)

## Built on by (most cited first, 17 of 17)

- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis](https://mlsanity.com/paper/2410.07155/) (2024)
- [Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuning](https://mlsanity.com/paper/2503.00306/) (2025)
- [Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers](https://mlsanity.com/paper/2406.18400/) (2024)
- [Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editing](https://mlsanity.com/paper/2408.12456/) (2024)
- [UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models](https://mlsanity.com/paper/2505.14679/) (2025)
- [Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundation](https://mlsanity.com/paper/2510.00625/) (2025)
- [MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs](https://mlsanity.com/paper/2602.10965/) (2026)
- [REPAIR: Robust Editing via Progressive Adaptive Intervention and Reintegration](https://mlsanity.com/paper/2510.01879/) (2025)
- [NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database](https://mlsanity.com/paper/2507.18028/) (2025)
- [Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning](https://mlsanity.com/paper/2510.13909/) (2025)
- [FiLoRA: Focus-and-Ignore LoRA for Controllable Feature Reliance](https://mlsanity.com/paper/2602.02060/) (2026)
- [MoKus: Leveraging Cross-Modal Knowledge Transfer for Knowledge-Aware Concept Customization](https://mlsanity.com/paper/2603.12743/) (2026)
- [BetaEdit: Null-Space Constrained Sequential Model Editing](https://mlsanity.com/paper/2605.09285/) (2026)
- [MetaKE: Meta-Learning for Knowledge Editing Toward a Better Accuracy-Editability Trade-off](https://mlsanity.com/paper/2603.12677/) (2026)
- [Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment](https://mlsanity.com/paper/2605.23780/) (2026)
- [Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing](https://mlsanity.com/paper/2608.11660/) (2026)

## Abstract (arXiv)

Knowledge editing for large language models can offer an efficient solution to alter a model's behavior without negatively impacting the overall performance. However, the current approaches encounter issues with limited generalizability across tasks, necessitating one distinct editor for each task, significantly hindering the broader applications. To address this, we take the first step to analyze the multi-task generalization issue in knowledge editing. Specifically, we develop an instruction-based editing technique, termed InstructEdit, which facilitates the editor's adaptation to various task performances simultaneously using simple instructions. With only one unified editor for each LLM, we empirically demonstrate that InstructEdit can improve the editor's control, leading to an average 14.86% increase in Reliability in multi-task editing setting. Furthermore, experiments involving holdout unseen task illustrate that InstructEdit consistently surpass previous strong baselines. To further investigate the underlying mechanisms of instruction-based knowledge editing, we analyze the principal components of the editing gradient directions, which unveils that instructions can help control optimization direction with stronger OOD generalization. Code and datasets are available in https://github.com/zjunlp/EasyEdit.