mlsanityData to 7 Oct 2026

Editing Large Language Models: Problems, Methods, and Opportunities

arXiv 2305.13172 ↗First posted 22 May 2023cs.CL cs.AI cs.CVcited by 367

Not on the map. Why?

Cited by 35 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the ability to train capable LLMs, the methodology for maintaining their relevancy and rectifying errors remains elusive. To this end, the past few years have witnessed a surge in techniques for editing LLMs, the objective of which is to efficiently alter the behavior of LLMs within a specific domain without negatively impacting performance across other inputs. This paper embarks on a deep exploration of the problems, methods, and opportunities related to model editing for LLMs. In particular, we provide an exhaustive overview of the task definition and challenges associated with model editing, along with an in-depth empirical analysis of the most progressive methods currently at our disposal. We also build a new benchmark dataset to facilitate a more robust evaluation and pinpoint enduring issues intrinsic to existing techniques. Our objective is to provide valuable insights into the effectiveness and feasibility of each editing technique, thereby assisting the community in making informed decisions on the selection of the most appropriate method for a specific task or context. Code and datasets are available at https://github.com/zjunlp/EasyEdit.

Key facts. Cited by 367 papers in the map (data through 7 Oct 2026). Builds on 56 of its 94 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

56 of 94 references.

References · 56

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  11. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  12. 2023A Survey of Large Language Modelscited by 2,537
38 more
  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2022Editing Models with Task Arithmeticcited by 1,076
  5. 2022Mass-Editing Memory in a Transformercited by 976
  6. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  7. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  8. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  9. 2021Editing Factual Knowledge in Language Modelscited by 548
  10. 2023Erasing Concepts from Diffusion Modelscited by 530
  11. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  12. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  13. 2021Fast Model Editing at Scalecited by 491
  14. 2022Memory-Based Model Editing at Scalecited by 443
  15. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  16. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
  17. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  18. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  19. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  20. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  21. 2020Modifying Memories in Transformer Modelscited by 222
  22. 2021Fast Yet Effective Machine Unlearningcited by 209
  23. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 196
  24. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 170
  25. 2020Editable Neural Networkscited by 154
  26. 2020Continual Lifelong Learning in Natural Language Processing: A Surveycited by 152
  27. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 149
  28. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  29. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 124
  30. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92
  31. 2023Safety Assessment of Chinese Large Language Modelscited by 82
  32. 2023Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledgecited by 66
  33. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  34. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  35. 2022PUMA: Performance Unchanged Model Augmentation for Training Data Removalcited by 56
  36. 2022Kformer: Knowledge Injection in Transformer Feed-Forward Layerscited by 37
  37. 2023In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPTcited by 35
  38. 2022Fixing Model Bugs with Natural Language Patchescited by 32

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 32; 2024: 163; 2025: 109; 2026: 63.

367 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  4. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  5. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  6. 2023Challenges and Applications of Large Language Modelscited by 251
  7. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  8. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  9. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  10. 2025From RAG to Memory: Non-Parametric Continual Learning for Large Language Modelscited by 223
  11. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  12. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
38 more
  1. 2023PMET: Precise Model Editing in a Transformercited by 170
  2. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  3. 2024Continual Learning for Large Language Models: A Surveycited by 144
  4. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  5. 2025MemOS: A Memory OS for AI Systemcited by 129
  6. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  7. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Eraserscited by 118
  8. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  9. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  10. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 101
  11. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  12. 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
  13. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 92
  14. 2024MEMORYLLM: Towards Self-Updatable Large Language Modelscited by 89
  15. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  16. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  17. 2023MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRAcited by 80
  18. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  19. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  20. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  21. 2023Massive Editing for Large Language Models via Meta Learningcited by 62
  22. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  23. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyondcited by 59
  24. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  25. 2024A Unified Framework for Model Editingcited by 54
  26. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  27. 2024Larimar: Large Language Models with Episodic Memory Controlcited by 53
  28. 2024Rebuilding ROME : Resolving Model Collapse during Sequential Model Editingcited by 50
  29. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  30. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  31. 2024Risks of AI Scientists: Prioritizing Safeguarding Over Autonomycited by 49
  32. 2024What does the Knowledge Neuron Thesis Have to do with Knowledge?cited by 49
  33. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  34. 2023Learning from models beyond fine-tuningcited by 47
  35. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAGcited by 47
  36. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  37. 2023Can We Edit Multimodal Large Language Models?cited by 44
  38. 2024WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editingcited by 40

50 most cited of 367 All 367 (JSON)