# AKEW: Assessing Knowledge Editing in the Wild

arXiv: https://arxiv.org/abs/2402.18909 · HTML: https://mlsanity.com/paper/2402.18909/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 30 of 32)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Continual Pre-training of Language Models](https://mlsanity.com/paper/2302.03241/) (2023)
- [Knowledge Editing for Large Language Models: A Survey](https://mlsanity.com/paper/2310.16218/) (2023)
- [Towards Continual Knowledge Learning of Language Models](https://mlsanity.com/paper/2110.03215/) (2021)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [Editable Neural Networks](https://mlsanity.com/paper/2004.00345/) (2020)
- [Fact-Checking Complex Claims with Program-Guided Reasoning](https://mlsanity.com/paper/2305.12744/) (2023)
- [Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases](https://mlsanity.com/paper/2106.09231/) (2021)
- [Calibrating Factual Knowledge in Pretrained Language Models](https://mlsanity.com/paper/2210.03329/) (2022)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing](https://mlsanity.com/paper/2402.10987/) (2024)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)

## Built on by (most cited first, 30 of 36)

- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [AnyEdit: Edit Any Knowledge Encoded in Language Models](https://mlsanity.com/paper/2502.05628/) (2025)
- [AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge](https://mlsanity.com/paper/2412.13670/) (2024)
- [A Survey of Recent Backdoor Attacks and Defenses in Large Language Models](https://mlsanity.com/paper/2406.06852/) (2024)
- [Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate](https://mlsanity.com/paper/2410.22086/) (2024)
- [The Mirage of Model Editing: Revisiting Evaluation in the Wild](https://mlsanity.com/paper/2502.11177/) (2025)
- [Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings](https://mlsanity.com/paper/2506.04997/) (2025)
- [Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models](https://mlsanity.com/paper/2405.15349/) (2024)
- [MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering](https://mlsanity.com/paper/2412.15540/) (2024)
- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models](https://mlsanity.com/paper/2505.14679/) (2025)
- [Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detection](https://mlsanity.com/paper/2506.08477/) (2025)
- [DocMEdit: Towards Document-Level Model Editing](https://mlsanity.com/paper/2505.19572/) (2025)
- [Is Fine-Tuning an Effective Solution? Reassessing Knowledge Editing for Unstructured Data](https://mlsanity.com/paper/2506.09672/) (2025)
- [Curriculum Demonstration Selection for In-Context Learning](https://mlsanity.com/paper/2411.18126/) (2024)
- [μKE: Matryoshka Unstructured Knowledge Editing of Large Language Models](https://mlsanity.com/paper/2504.01196/) (2025)
- [Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editing](https://mlsanity.com/paper/2505.12636/) (2025)
- [Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels](https://mlsanity.com/paper/2409.12425/) (2024)
- [CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Base](https://mlsanity.com/paper/2502.12591/) (2025)
- [Uncovering Context Reliance in Unstructured Knowledge Editing](https://mlsanity.com/paper/2602.19043/) (2026)
- [CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMs](https://mlsanity.com/paper/2409.05806/) (2024)
- [Dynamic Retriever for In-Context Knowledge Editing via Policy Optimization](https://mlsanity.com/paper/2510.21059/) (2025)
- [Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs](https://mlsanity.com/paper/2511.20892/) (2025)
- [LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs](https://mlsanity.com/paper/2511.14774/) (2025)
- [FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing](https://mlsanity.com/paper/2604.12559/) (2026)
- [SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation](https://mlsanity.com/paper/2505.14419/) (2025)
- [ScEdit: Script-based Assessment of Knowledge Editing](https://mlsanity.com/paper/2505.23291/) (2025)
- [The Model Agreed, But Didn't Learn: Diagnosing Surface Compliance in Large Language Models](https://mlsanity.com/paper/2604.05995/) (2026)
- [ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systems](https://mlsanity.com/paper/2503.20756/) (2025)
- [EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledge](https://mlsanity.com/paper/2507.03617/) (2025)

## Abstract (arXiv)

Knowledge editing injects knowledge updates into language models to keep them correct and up-to-date. However, its current evaluations deviate significantly from practice: their knowledge updates solely consist of structured facts derived from meticulously crafted datasets, instead of practical sources -- unstructured texts like news articles, and they often overlook practical real-world knowledge updates. To address these issues, in this paper we propose AKEW (Assessing Knowledge Editing in the Wild), a new practical benchmark for knowledge editing. AKEW fully covers three editing settings of knowledge updates: structured facts, unstructured texts as facts, and extracted triplets. It further introduces new datasets featuring both counterfactual and real-world knowledge updates. Through extensive experiments, we demonstrate the considerable gap between state-of-the-art knowledge-editing methods and practical scenarios. Our analyses further highlight key insights to motivate future research for practical knowledge editing.