# A Survey on Knowledge Editing of Neural Networks

arXiv: https://arxiv.org/abs/2310.19704 · HTML: https://mlsanity.com/paper/2310.19704/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 86 of its 134 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 30 of 86)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks](https://mlsanity.com/paper/1702.01135/) (2017)

## Built on by (most cited first, 30 of 34)

- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Detoxifying Large Language Models via Knowledge Editing](https://mlsanity.com/paper/2403.14472/) (2024)
- [Knowledge Mechanisms in Large Language Models: A Survey and Perspective](https://mlsanity.com/paper/2407.15017/) (2024)
- [Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing](https://mlsanity.com/paper/2410.06331/) (2024)
- [DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMs](https://mlsanity.com/paper/2404.08700/) (2024)
- [Editing Conceptual Knowledge for Large Language Models](https://mlsanity.com/paper/2403.06259/) (2024)
- [Should We Really Edit Language Models? On the Evaluation of Edited Language Models](https://mlsanity.com/paper/2410.18785/) (2024)
- [InstructEdit: Instruction-based Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2402.16123/) (2024)
- [Has this Fact been Edited? Detecting Knowledge Edits in Language Models](https://mlsanity.com/paper/2405.02765/) (2024)
- [Dense Passage Retrieval: Is it Retrieving?](https://mlsanity.com/paper/2402.11035/) (2024)
- [Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models](https://mlsanity.com/paper/2406.01436/) (2024)
- [LLMs as Repositories of Factual Knowledge: Limitations and Solutions](https://mlsanity.com/paper/2501.12774/) (2025)
- [Revealing and Mitigating Over-Attention in Knowledge Editing](https://mlsanity.com/paper/2502.14838/) (2025)
- [TAXI: Evaluating Categorical Knowledge Editing for Language Models](https://mlsanity.com/paper/2404.15004/) (2024)
- [Reasons and Solutions for the Decline in Model Performance after Editing](https://mlsanity.com/paper/2410.23843/) (2024)
- [K-Edit: Language Model Editing with Contextual Knowledge Awareness](https://mlsanity.com/paper/2502.10626/) (2025)
- [Composable Interventions for Language Models](https://mlsanity.com/paper/2407.06483/) (2024)
- [Rethinking Residual Distribution in Locate-then-Edit Model Editing](https://mlsanity.com/paper/2502.03748/) (2025)
- [ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall](https://mlsanity.com/paper/2510.07896/) (2025)
- [MolEdit: Knowledge Editing for Multimodal Molecule Language Models](https://mlsanity.com/paper/2511.12770/) (2025)
- [Editing Across Languages: A Survey of Multilingual Knowledge Editing](https://mlsanity.com/paper/2505.14393/) (2025)
- [Prototype-Grounded Concept Models for Verifiable Concept Alignment](https://mlsanity.com/paper/2604.16076/) (2026)
- [One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them](https://mlsanity.com/paper/2605.28839/) (2026)
- [Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge](https://mlsanity.com/paper/2406.02882/) (2024)
- [REMISVFU: Vertical Federated Unlearning via Representation Misdirection for Intermediate Output Feature](https://mlsanity.com/paper/2512.10348/) (2025)
- [Persuasion Tokens for Editing Factual Knowledge in LLMs](https://mlsanity.com/paper/2601.16781/) (2026)
- [Towards Reliable, Generalizable, and Specific In-Context Knowledge Editing via Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2608.25100/) (2026)
- [A Dual-Axis Taxonomy of Knowledge Editing for LLMs: From Mechanisms to Functions](https://mlsanity.com/paper/2508.08795/) (2025)

## Abstract (arXiv)

Deep neural networks are becoming increasingly pervasive in academia and industry, matching and surpassing human performance on a wide variety of fields and related tasks. However, just as humans, even the largest artificial neural networks make mistakes, and once-correct predictions can become invalid as the world progresses in time. Augmenting datasets with samples that account for mistakes or up-to-date information has become a common workaround in practical applications. However, the well-known phenomenon of catastrophic forgetting poses a challenge in achieving precise changes in the implicitly memorized knowledge of neural network parameters, often requiring a full model re-training to achieve desired behaviors. That is expensive, unreliable, and incompatible with the current trend of large self-supervised pre-training, making it necessary to find more efficient and effective methods for adapting neural network models to changing data. To address this need, knowledge editing is emerging as a novel area of research that aims to enable reliable, data-efficient, and fast changes to a pre-trained target model, without affecting model behaviors on previously learned tasks. In this survey, we provide a brief review of this recent artificial intelligence field of research. We first introduce the problem of editing neural networks, formalize it in a common framework and differentiate it from more notorious branches of research such as continuous learning. Next, we provide a review of the most relevant knowledge editing approaches and datasets proposed so far, grouping works under four different families: regularization techniques, meta-learning, direct model editing, and architectural strategies. Finally, we outline some intersections with other fields of research and potential directions for future works.