mlsanityData to 7 Oct 2026

Knowledge Editing for Large Language Models: A Survey

arXiv 2310.16218 ↗First posted 24 Oct 2023cs.CL cs.AIcited by 189

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large language models (LLMs) have recently transformed both the academic and industrial landscapes due to their remarkable capacity to understand, analyze, and generate texts based on their vast knowledge and reasoning ability. Nevertheless, one major drawback of LLMs is their substantial computational cost for pre-training due to their unprecedented amounts of parameters. The disadvantage is exacerbated when new knowledge frequently needs to be introduced into the pre-trained model. Therefore, it is imperative to develop effective and efficient techniques to update pre-trained LLMs. Traditional methods encode new knowledge in pre-trained LLMs through direct fine-tuning. However, naively re-training LLMs can be computationally intensive and risks degenerating valuable pre-trained knowledge irrelevant to the update in the model. Recently, Knowledge-based Model Editing (KME) has attracted increasing attention, which aims to precisely modify the LLMs to incorporate specific knowledge, without negatively influencing other irrelevant knowledge. In this survey, we aim to provide a comprehensive and in-depth overview of recent advances in the field of KME. We first introduce a general formulation of KME to encompass different KME strategies. Afterward, we provide an innovative taxonomy of KME techniques based on how the new knowledge is introduced into pre-trained LLMs, and investigate existing KME strategies while analyzing key insights, advantages, and limitations of methods from each category. Moreover, representative metrics, datasets, and applications of KME are introduced accordingly. Finally, we provide an in-depth analysis regarding the practicality and remaining challenges of KME and suggest promising research directions for further advancement in this field.

Key facts. Cited by 189 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 113 of its 170 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

113 of 170 references.

References · 113

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  10. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  11. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  12. 2022Generative Adversarial Networkscited by 5,603
38 more
  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  7. 2015VQA: Visual Question Answeringcited by 2,489
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  13. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  14. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  15. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  16. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  17. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  18. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  19. 2016HyperNetworkscited by 1,003
  20. 2022Red Teaming Language Models with Language Modelscited by 983
  21. 2022Mass-Editing Memory in a Transformercited by 976
  22. 2019How Can We Know What Language Models Know?cited by 953
  23. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  24. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  25. 2021Robust fine-tuning of zero-shot modelscited by 785
  26. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  27. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  28. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  29. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  30. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  31. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  32. 2020Recipes for building an open-domain chatbotcited by 593
  33. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  34. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  35. 2021Editing Factual Knowledge in Language Modelscited by 548
  36. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  37. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  38. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457

50 most cited of 113 All 113 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 74; 2025: 68; 2026: 45.

189 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  3. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  4. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  5. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  6. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  7. 2024BadEdit: Backdooring large language models by model editingcited by 84
  8. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 84
  9. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  10. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  11. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  12. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 65
38 more
  1. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 60
  2. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  3. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  4. 2024To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Modelscited by 42
  5. 2024Multi-hop Question Answering under Temporal Knowledge Editingcited by 36
  6. 2024AKEW: Assessing Knowledge Editing in the Wildcited by 35
  7. 2024VLKEB: A Large Vision-Language Model Knowledge Editing Benchmarkcited by 30
  8. 2025SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signalscited by 29
  9. 2024Knowledge Graph-Enhanced Large Language Models via Path Selectioncited by 28
  10. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  11. 2025Taming Knowledge Conflicts in Language Modelscited by 27
  12. 2024Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Taskscited by 24
  13. 2024Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?cited by 24
  14. 2025The Mirage of Model Editing: Revisiting Evaluation in the Wildcited by 24
  15. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  16. 2024Can Editing LLMs Inject Harm?cited by 22
  17. 2024Configurable Foundation Models: Building LLMs from a Modular Perspectivecited by 22
  18. 2024Editing Conceptual Knowledge for Large Language Modelscited by 21
  19. 2024A Survey on the Honesty of Large Language Modelscited by 21
  20. 2024Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Modelscited by 20
  21. 2024RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuningcited by 20
  22. 2024Prospect Personalized Recommendation on Large Language Model-based Agent Platformcited by 19
  23. 2025Benchmarking Retrieval-Augmented Generation for Chemistrycited by 19
  24. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  25. 2024CodeUpdateArena: Benchmarking Knowledge Editing on API Updatescited by 18
  26. 2024Should We Really Edit Language Models? On the Evaluation of Edited Language Modelscited by 18
  27. 2024InstructEdit: Instruction-based Knowledge Editing for Large Language Modelscited by 17
  28. 2024Knowledge Editing in Language Models via Adapted Direct Preference Optimizationcited by 17
  29. 2024CEB: Compositional Evaluation Benchmark for Fairness in Large Language Modelscited by 17
  30. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 17
  31. 2023Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Modelscited by 15
  32. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Modelscited by 15
  33. 2024Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Modelscited by 14
  34. 2024Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuningcited by 14
  35. 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awarenesscited by 13
  36. 2023Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Interventioncited by 12
  37. 2024LeKUBE: A Legal Knowledge Update BEnchmarkcited by 12
  38. 2025CoME: An Unlearning-based Approach to Conflict-free Model Editingcited by 12

50 most cited of 189 All 189 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking and Rethinking Knowledge Editing for Large Language Models
  2. 2025Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning
  3. 2024Learning to Edit: Aligning LLMs with Knowledge Editing
  4. 2023Retrieval-augmented Multilingual Knowledge Editing
  5. 2025Aligning Language Models with Real-time Knowledge Editing
  6. 2025Massive Editing for Large Language Models Based on Dynamic Weight Generation
  7. 2024MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptation
  8. 2024Uncovering Overfitting in Large Language Model Editing