mlsanityData to 7 Oct 2026

AKEW: Assessing Knowledge Editing in the Wild

arXiv 2402.18909 ↗First posted 29 Feb 2024cs.CL cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

Knowledge editing injects knowledge updates into language models to keep them correct and up-to-date. However, its current evaluations deviate significantly from practice: their knowledge updates solely consist of structured facts derived from meticulously crafted datasets, instead of practical sources -- unstructured texts like news articles, and they often overlook practical real-world knowledge updates. To address these issues, in this paper we propose AKEW (Assessing Knowledge Editing in the Wild), a new practical benchmark for knowledge editing. AKEW fully covers three editing settings of knowledge updates: structured facts, unstructured texts as facts, and extracted triplets. It further introduces new datasets featuring both counterfactual and real-world knowledge updates. Through extensive experiments, we demonstrate the considerable gap between state-of-the-art knowledge-editing methods and practical scenarios. Our analyses further highlight key insights to motivate future research for practical knowledge editing.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

32 of 50 references.

References · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  6. 2022Mass-Editing Memory in a Transformercited by 976
  7. 2021Editing Factual Knowledge in Language Modelscited by 548
  8. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  9. 2022Memory-Based Model Editing at Scalecited by 443
  10. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  11. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  12. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
20 more
  1. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 296
  2. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  3. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  4. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  5. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  6. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  7. 2020Modifying Memories in Transformer Modelscited by 222
  8. 2023Continual Pre-training of Language Modelscited by 194
  9. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  10. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  11. 2023PMET: Precise Model Editing in a Transformercited by 170
  12. 2020Editable Neural Networkscited by 154
  13. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  14. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  15. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92
  16. 2023Effective Neural Topic Modeling with Embedding Clustering Regularizationcited by 46
  17. 2024WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editingcited by 40
  18. 2022Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learningcited by 38
  19. 2023DocTER: Evaluating Document-based Knowledge Editingcited by 26
  20. 2023Towards the TopMost: A Topic Modeling System Toolkitcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 17; 2026: 8.

35 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  2. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 74
  3. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  4. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  5. 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning ratecited by 29
  6. 2025Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddingscited by 26
  7. 2025The Mirage of Model Editing: Revisiting Evaluation in the Wildcited by 24
  8. 2024Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Modelscited by 18
  9. 2024MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answeringcited by 17
  10. 2024Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusioncited by 16
  11. 2025UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Modelscited by 7
  12. 2025Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detectioncited by 6
23 more
  1. 2025Is Fine-Tuning an Effective Solution? Reassessing Knowledge Editing for Unstructured Datacited by 6
  2. 2024Curriculum Demonstration Selection for In-Context Learningcited by 5
  3. 2025Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editingcited by 5
  4. 2025DocMEdit: Towards Document-Level Model Editingcited by 5
  5. 2024Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labelscited by 4
  6. 2025CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Basecited by 4
  7. 2025μKE: Matryoshka Unstructured Knowledge Editing of Large Language Modelscited by 4
  8. 2024CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMscited by 3
  9. 2025Dynamic Retriever for In-Context Knowledge Editing via Policy Optimizationcited by 3
  10. 2026Uncovering Context Reliance in Unstructured Knowledge Editingcited by 3
  11. 2025LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMscited by 2
  12. 2025Representation Interventions Enable Lifelong Knowledge Memory Control in LLMscited by 2
  13. 2025SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotationcited by 1
  14. 2025ScEdit: Script-based Assessment of Knowledge Editingcited by 1
  15. 2026FABLE: Fine-grained Fact Anchoring for Unstructured Model Editingcited by 1
  16. 2025ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systemscited by 0
  17. 2025EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledgecited by 0
  18. 2026MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrievalcited by 0
  19. 2026The Model Agreed, But Didn't Learn: Diagnosing Surface Compliance in Large Language Modelscited by 0
  20. 2026Implicit Reasoning Steering via Concept Chainingcited by 0
  21. 2026Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editingcited by 0
  22. 2026On Scope Classification and Current Knowledge-Editing Benchmarks: A Negative Result, with INLAY as a Gradient-Free Case Studycited by 0
  23. 2026ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspectivecited by 0