# Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models

arXiv: https://arxiv.org/abs/2405.15349 · HTML: https://mlsanity.com/paper/2405.15349/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 30 of 34)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models](https://mlsanity.com/paper/2301.04213/) (2023)
- [Transformer-Patcher: One Mistake worth One Neuron](https://mlsanity.com/paper/2301.09785/) (2023)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models](https://mlsanity.com/paper/2308.07269/) (2023)
- [Combating Misinformation in the Age of LLMs: Opportunities and Challenges](https://mlsanity.com/paper/2311.05656/) (2023)
- [Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks](https://mlsanity.com/paper/2304.14732/) (2023)
- [Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge](https://mlsanity.com/paper/2305.01651/) (2023)
- [Can We Edit Multimodal Large Language Models?](https://mlsanity.com/paper/2310.08475/) (2023)
- [AKEW: Assessing Knowledge Editing in the Wild](https://mlsanity.com/paper/2402.18909/) (2024)
- [DocTER: Evaluating Document-based Knowledge Editing](https://mlsanity.com/paper/2308.09954/) (2023)
- [Long-form evaluation of model editing](https://mlsanity.com/paper/2402.09394/) (2024)
- [EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries](https://mlsanity.com/paper/2402.11324/) (2024)
- [Retrieval-augmented Multilingual Knowledge Editing](https://mlsanity.com/paper/2312.13040/) (2023)
- [Stable Knowledge Editing in Large Language Models](https://mlsanity.com/paper/2402.13048/) (2024)

## Built on by (most cited first, 19 of 19)

- [AnyEdit: Edit Any Knowledge Encoded in Language Models](https://mlsanity.com/paper/2502.05628/) (2025)
- [RLKD: Distilling LLMs' Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2505.16142/) (2025)
- [UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models](https://mlsanity.com/paper/2505.14679/) (2025)
- [LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models](https://mlsanity.com/paper/2409.13054/) (2024)
- [SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models](https://mlsanity.com/paper/2510.16917/) (2025)
- [Circular Reasoning: Understanding Self-Reinforcing Loops in Large Reasoning Models](https://mlsanity.com/paper/2601.05693/) (2026)
- [ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing](https://mlsanity.com/paper/2412.12821/) (2024)
- [ThinkEval: Practical Evaluation of Knowledge Leakage in LLM Editing using Thought-based Knowledge Graphs](https://mlsanity.com/paper/2506.01386/) (2025)
- [MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs](https://mlsanity.com/paper/2602.10965/) (2026)
- [Uncovering Context Reliance in Unstructured Knowledge Editing](https://mlsanity.com/paper/2602.19043/) (2026)
- [Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMs](https://mlsanity.com/paper/2510.09885/) (2025)
- [Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing](https://mlsanity.com/paper/2601.15686/) (2026)
- [Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis](https://mlsanity.com/paper/2602.20207/) (2026)
- [Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates](https://mlsanity.com/paper/2605.16686/) (2026)
- [Modality-Decoupled Online Recursive Editing](https://mlsanity.com/paper/2605.20273/) (2026)
- [Distributed Multi-Layer Editing for Rule-Level Knowledge in Large Language Models](https://mlsanity.com/paper/2604.08284/) (2026)
- [Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment](https://mlsanity.com/paper/2605.23780/) (2026)
- [ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective](https://mlsanity.com/paper/2609.33534/) (2026)
- [Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing](https://mlsanity.com/paper/2610.02772/) (2026)

## Abstract (arXiv)

Recent knowledge editing methods have primarily focused on modifying structured knowledge in large language models. However, this task setting overlooks the fact that a significant portion of real-world knowledge is stored in an unstructured format, characterized by long-form content, noise, and a complex yet comprehensive nature. Techniques like "local layer key-value storage" and "term-driven optimization", as used in previous methods like MEMIT, are not effective for handling unstructured knowledge. To address these challenges, we propose a novel Unstructured Knowledge Editing method, namely UnKE, which extends previous assumptions in the layer dimension and token dimension. Firstly, in the layer dimension, we propose non-local block key-value storage to replace local layer key-value storage, increasing the representation ability of key-value pairs and incorporating attention layer knowledge. Secondly, in the token dimension, we replace "term-driven optimization" with "cause-driven optimization", which edits the last token directly while preserving context, avoiding the need to locate terms and preventing the loss of context information. Results on newly proposed unstructured knowledge editing dataset (UnKEBench) and traditional structured datasets demonstrate that UnKE achieves remarkable performance, surpassing strong baselines. In addition, UnKE has robust batch editing and sequential editing capabilities.