mlsanityData to 14 Oct 2026

InstructEdit: Instruction-based Knowledge Editing for Large Language Models

arXiv 2402.16123 ↗First posted 25 Feb 2024cs.CL cs.AI cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Knowledge editing for large language models can offer an efficient solution to alter a model's behavior without negatively impacting the overall performance. However, the current approaches encounter issues with limited generalizability across tasks, necessitating one distinct editor for each task, significantly hindering the broader applications. To address this, we take the first step to analyze the multi-task generalization issue in knowledge editing. Specifically, we develop an instruction-based editing technique, termed InstructEdit, which facilitates the editor's adaptation to various task performances simultaneously using simple instructions. With only one unified editor for each LLM, we empirically demonstrate that InstructEdit can improve the editor's control, leading to an average 14.86% increase in Reliability in multi-task editing setting. Furthermore, experiments involving holdout unseen task illustrate that InstructEdit consistently surpass previous strong baselines. To further investigate the underlying mechanisms of instruction-based knowledge editing, we analyze the principal components of the editing gradient directions, which unveils that instructions can help control optimization direction with stronger OOD generalization. Code and datasets are available in https://github.com/zjunlp/EasyEdit.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 54 of its 63 references that are in the map. First posted on arXiv 25 Feb 2024.

Builds on

54 of 63 references.

References · 54

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2022Editing Models with Task Arithmeticcited by 1,120
  10. 2022Mass-Editing Memory in a Transformercited by 1,001
  11. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 601
38 more
  1. 2021Editing Factual Knowledge in Language Modelscited by 553
  2. 2023Erasing Concepts from Diffusion Modelscited by 539
  3. 2021Fast Model Editing at Scalecited by 498
  4. 2022Memory-Based Model Editing at Scalecited by 447
  5. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  6. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  7. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  8. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  9. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  10. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  11. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
  12. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  13. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  14. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  15. 2023PMET: Precise Model Editing in a Transformercited by 173
  16. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  17. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 103
  18. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 98
  19. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 93
  20. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92
  21. 2023MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRAcited by 80
  22. 2023Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledgecited by 67
  23. 2023Massive Editing for Large Language Models via Meta Learningcited by 63
  24. 2023Can We Edit Multimodal Large Language Models?cited by 44
  25. 2023Unveiling the Pitfalls of Knowledge Editing for Large Language Modelscited by 43
  26. 2023Finding and Editing Multi-Modal Neurons in Pre-Trained Transformerscited by 36
  27. 2023A Survey on Knowledge Editing of Neural Networkscited by 34
  28. 2023Emptying the Ocean with a Spoon: Should We Edit Models?cited by 27
  29. 2024Large Language Models Relearn Removed Conceptscited by 27
  30. 2022Language Anisotropic Cross-Lingual Model Editingcited by 26
  31. 2023DocTER: Evaluating Document-based Knowledge Editingcited by 26
  32. 2023Editing Language Model-based Knowledge Graph Embeddingscited by 23
  33. 2023DUnE: Dataset for Unified Editingcited by 19
  34. 2023History Matters: Temporal Knowledge Editing in Large Language Modelcited by 17
  35. 2023Assessing Knowledge Editing in Language Models via Relation Perspectivecited by 15
  36. 2023Evaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awarenesscited by 14
  37. 2024Neighboring Perturbations of Knowledge Editing on Large Language Modelscited by 10
  38. 2024MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model Editingcited by 9

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 7.

17 papers.

Sort:

By area:

  1. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
  2. 2024Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesiscited by 19
  3. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 19
  4. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 15
  5. 2024Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editingcited by 9
  6. 2025UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Modelscited by 9
  7. 2025Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundationcited by 7
  8. 2026MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMscited by 4
  9. 2025REPAIR: Robust Editing via Progressive Adaptive Intervention and Reintegrationcited by 3
  10. 2025NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Databasecited by 2
  11. 2025Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoningcited by 1
  12. 2026FiLoRA: Focus-and-Ignore LoRA for Controllable Feature Reliancecited by 1
5 more
  1. 2026MoKus: Leveraging Cross-Modal Knowledge Transfer for Knowledge-Aware Concept Customizationcited by 1
  2. 2026BetaEdit: Null-Space Constrained Sequential Model Editingcited by 1
  3. 2026MetaKE: Meta-Learning for Knowledge Editing Toward a Better Accuracy-Editability Trade-offcited by 0
  4. 2026Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignmentcited by 0
  5. 2026Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editingcited by 0