# Commonsense Knowledge Editing Based on Free-Text in LLMs

arXiv: https://arxiv.org/abs/2410.23844 · HTML: https://mlsanity.com/paper/2410.23844/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [Editing Common Sense in Transformers](https://mlsanity.com/paper/2305.14956/) (2023)
- [Modeling Event Plausibility with Consistent Conceptual Abstraction](https://mlsanity.com/paper/2104.10247/) (2021)
- [EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries](https://mlsanity.com/paper/2402.11324/) (2024)
- [Assessing Knowledge Editing in Language Models via Relation Perspective](https://mlsanity.com/paper/2311.09053/) (2023)
- [KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models](https://mlsanity.com/paper/2309.16535/) (2023)

## Built on by (most cited first, 16 of 16)

- [AnyEdit: Edit Any Knowledge Encoded in Language Models](https://mlsanity.com/paper/2502.05628/) (2025)
- [Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models](https://mlsanity.com/paper/2502.15910/) (2025)
- [GeoEdit: Geometric Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2502.19953/) (2025)
- [Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Models](https://mlsanity.com/paper/2601.09445/) (2026)
- [SetKE: Knowledge Editing for Knowledge Elements Overlap](https://mlsanity.com/paper/2504.20972/) (2025)
- [Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning](https://mlsanity.com/paper/2605.05909/) (2026)
- [EvoEdit: Lifelong Free-Text Knowledge Editing through Latent Perturbation Augmentation and Knowledge-driven Parameter Fusion](https://mlsanity.com/paper/2512.04545/) (2025)
- [FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing](https://mlsanity.com/paper/2604.12559/) (2026)
- [AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise](https://mlsanity.com/paper/2606.01053/) (2026)
- [Capability Localization: Capabilities Can be Localized rather than Individual Knowledge](https://mlsanity.com/paper/2502.20992/) (2025)
- [ScEdit: Script-based Assessment of Knowledge Editing](https://mlsanity.com/paper/2505.23291/) (2025)
- [Robust MLLM Unlearning via Visual Knowledge Distillation](https://mlsanity.com/paper/2512.11325/) (2025)
- [Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs](https://mlsanity.com/paper/2603.09434/) (2026)
- [KEditVis: A Visual Analytics System for Knowledge Editing of Large Language Models](https://mlsanity.com/paper/2603.29689/) (2026)
- [A Dual-Axis Taxonomy of Knowledge Editing for LLMs: From Mechanisms to Functions](https://mlsanity.com/paper/2508.08795/) (2025)
- [ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective](https://mlsanity.com/paper/2609.33534/) (2026)

## Abstract (arXiv)

Knowledge editing technology is crucial for maintaining the accuracy and timeliness of large language models (LLMs) . However, the setting of this task overlooks a significant portion of commonsense knowledge based on free-text in the real world, characterized by broad knowledge scope, long content and non instantiation. The editing objects of previous methods (e.g., MEMIT) were single token or entity, which were not suitable for commonsense knowledge in free-text form. To address the aforementioned challenges, we conducted experiments from two perspectives: knowledge localization and knowledge editing. Firstly, we introduced Knowledge Localization for Free-Text(KLFT) method, revealing the challenges associated with the distribution of commonsense knowledge in MLP and Attention layers, as well as in decentralized distribution. Next, we propose a Dynamics-aware Editing Method(DEM), which utilizes a Dynamics-aware Module to locate the parameter positions corresponding to commonsense knowledge, and uses Knowledge Editing Module to update knowledge. The DEM method fully explores the potential of the MLP and Attention layers, and successfully edits commonsense knowledge based on free-text. The experimental results indicate that the DEM can achieve excellent editing performance.