mlsanityData to 14 Oct 2026

Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models

arXiv 2405.15349 ↗First posted 24 May 2024cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Recent knowledge editing methods have primarily focused on modifying structured knowledge in large language models. However, this task setting overlooks the fact that a significant portion of real-world knowledge is stored in an unstructured format, characterized by long-form content, noise, and a complex yet comprehensive nature. Techniques like "local layer key-value storage" and "term-driven optimization", as used in previous methods like MEMIT, are not effective for handling unstructured knowledge. To address these challenges, we propose a novel Unstructured Knowledge Editing method, namely UnKE, which extends previous assumptions in the layer dimension and token dimension. Firstly, in the layer dimension, we propose non-local block key-value storage to replace local layer key-value storage, increasing the representation ability of key-value pairs and incorporating attention layer knowledge. Secondly, in the token dimension, we replace "term-driven optimization" with "cause-driven optimization", which edits the last token directly while preserving context, avoiding the need to locate terms and preventing the loss of context information. Results on newly proposed unstructured knowledge editing dataset (UnKEBench) and traditional structured datasets demonstrate that UnKE achieves remarkable performance, surpassing strong baselines. In addition, UnKE has robust batch editing and sequential editing capabilities.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

34 of 48 references.

References · 34

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  6. 2022Mass-Editing Memory in a Transformercited by 1,001
  7. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  8. 2021Fast Model Editing at Scalecited by 498
  9. 2022Memory-Based Model Editing at Scalecited by 447
  10. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  11. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  12. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
22 more
  1. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  2. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  3. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  4. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  5. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  6. 2023PMET: Precise Model Editing in a Transformercited by 173
  7. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  8. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 98
  9. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 95
  10. 2023Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Taskscited by 73
  11. 2023Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledgecited by 67
  12. 2023Can We Edit Multimodal Large Language Models?cited by 44
  13. 2024AKEW: Assessing Knowledge Editing in the Wildcited by 36
  14. 2023DocTER: Evaluating Document-based Knowledge Editingcited by 26
  15. 2024Long-form evaluation of model editingcited by 23
  16. 2024EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundariescited by 16
  17. 2023Retrieval-augmented Multilingual Knowledge Editingcited by 13
  18. 2024Stable Knowledge Editing in Large Language Modelscited by 13
  19. 2023Cross-Lingual Knowledge Editing in Large Language Modelscited by 11
  20. 2024MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Modelscited by 11
  21. 2024Beyond Memorization: The Challenge of Random Memory Access in Language Modelscited by 8
  22. 2022IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selectioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 11.

19 papers.

Sort:

By area:

  1. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 76
  2. 2025RLKD: Distilling LLMs' Reasoning via Reinforcement Learningcited by 11
  3. 2025UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Modelscited by 9
  4. 2024LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Modelscited by 5
  5. 2025SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Modelscited by 5
  6. 2026Circular Reasoning: Understanding Self-Reinforcing Loops in Large Reasoning Modelscited by 5
  7. 2024ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editingcited by 4
  8. 2025ThinkEval: Practical Evaluation of Knowledge Leakage in LLM Editing using Thought-based Knowledge Graphscited by 4
  9. 2026MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMscited by 4
  10. 2026Uncovering Context Reliance in Unstructured Knowledge Editingcited by 4
  11. 2025Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMscited by 2
  12. 2026Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editingcited by 1
7 more
  1. 2026Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysiscited by 1
  2. 2026Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updatescited by 1
  3. 2026Modality-Decoupled Online Recursive Editingcited by 1
  4. 2026Distributed Multi-Layer Editing for Rule-Level Knowledge in Large Language Modelscited by 0
  5. 2026Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignmentcited by 0
  6. 2026ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspectivecited by 0
  7. 2026Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editingcited by 0