# AnyEdit: Edit Any Knowledge Encoded in Language Models

arXiv: https://arxiv.org/abs/2502.05628 · HTML: https://mlsanity.com/paper/2502.05628/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 8 Feb 2025.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Transformer-Patcher: One Mistake worth One Neuron](https://mlsanity.com/paper/2301.09785/) (2023)
- [Orca-Math: Unlocking the potential of SLMs in Grade School Math](https://mlsanity.com/paper/2402.14830/) (2024)
- [Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue](https://mlsanity.com/paper/2401.04700/) (2024)
- [FlipAttack: Jailbreak LLMs via Flipping](https://mlsanity.com/paper/2410.02832/) (2024)
- [AKEW: Assessing Knowledge Editing in the Wild](https://mlsanity.com/paper/2402.18909/) (2024)
- [Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts](https://mlsanity.com/paper/2405.11613/) (2024)
- [Can Knowledge Editing Really Correct Hallucinations?](https://mlsanity.com/paper/2410.16251/) (2024)
- [Can Editing LLMs Inject Harm?](https://mlsanity.com/paper/2407.20224/) (2024)
- [Neuron-Level Sequential Editing for Large Language Models](https://mlsanity.com/paper/2410.04045/) (2024)
- [Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models](https://mlsanity.com/paper/2405.15349/) (2024)
- [Commonsense Knowledge Editing Based on Free-Text in LLMs](https://mlsanity.com/paper/2410.23844/) (2024)
- [Structure Guided Large Language Model for SQL Generation](https://mlsanity.com/paper/2402.13284/) (2024)

## Built on by (most cited first, 30 of 76)

- [MemOS: A Memory OS for AI System](https://mlsanity.com/paper/2507.03724/) (2025)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editing](https://mlsanity.com/paper/2512.07826/) (2025)
- [OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation](https://mlsanity.com/paper/2509.03498/) (2025)
- [LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corpora](https://mlsanity.com/paper/2510.10114/) (2025)
- [When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.05690/) (2025)
- [Reinforced Lifelong Editing for Language Models](https://mlsanity.com/paper/2502.05759/) (2025)
- [FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.08938/) (2025)
- [EditThinker: Unlocking Iterative Reasoning for Any Image Editor](https://mlsanity.com/paper/2512.05965/) (2025)
- [Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms](https://mlsanity.com/paper/2505.20322/) (2025)
- [BiasEdit: Debiasing Stereotyped Language Models via Model Editing](https://mlsanity.com/paper/2503.08588/) (2025)
- [Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generation](https://mlsanity.com/paper/2508.03320/) (2025)
- [Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?](https://mlsanity.com/paper/2505.19855/) (2025)
- [OmniTry: Virtual Try-On Anything without Masks](https://mlsanity.com/paper/2508.13632/) (2025)
- [CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners](https://mlsanity.com/paper/2503.16356/) (2025)
- [UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models](https://mlsanity.com/paper/2505.14679/) (2025)
- [Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics](https://mlsanity.com/paper/2602.02343/) (2026)
- [When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations](https://mlsanity.com/paper/2505.24449/) (2025)
- [Fine-Grained Activation Steering: Steering Less, Achieving More](https://mlsanity.com/paper/2602.04428/) (2026)
- [ACE: Concept Editing in Diffusion Models without Performance Degradation](https://mlsanity.com/paper/2503.08116/) (2025)
- [Is Fine-Tuning an Effective Solution? Reassessing Knowledge Editing for Unstructured Data](https://mlsanity.com/paper/2506.09672/) (2025)
- [μKE: Matryoshka Unstructured Knowledge Editing of Large Language Models](https://mlsanity.com/paper/2504.01196/) (2025)
- [SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models](https://mlsanity.com/paper/2510.16917/) (2025)
- [Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse](https://mlsanity.com/paper/2601.11042/) (2026)
- [MMKU-Bench: A Multimodal Update Benchmark for Diverse Visual Knowledge](https://mlsanity.com/paper/2603.15117/) (2026)
- [Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editing](https://mlsanity.com/paper/2506.03490/) (2025)
- [POME: Post Optimization Model Edit via Muon-style Projection](https://mlsanity.com/paper/2510.06627/) (2025)
- [Are We Evaluating the Edit Locality of LLM Model Editing Properly?](https://mlsanity.com/paper/2601.17343/) (2026)
- [Uncovering Context Reliance in Unstructured Knowledge Editing](https://mlsanity.com/paper/2602.19043/) (2026)

## Abstract (arXiv)

Large language models (LLMs) often produce incorrect or outdated information, necessitating efficient and precise knowledge updates. Current model editing methods, however, struggle with long-form knowledge in diverse formats, such as poetry, code snippets, and mathematical derivations. These limitations arise from their reliance on editing a single token's hidden state, a limitation we term "efficacy barrier". To solve this, we propose AnyEdit, a new autoregressive editing paradigm. It decomposes long-form knowledge into sequential chunks and iteratively edits the key token in each chunk, ensuring consistent and accurate outputs. Theoretically, we ground AnyEdit in the Chain Rule of Mutual Information, showing its ability to update any knowledge within LLMs. Empirically, it outperforms strong baselines by 21.5% on benchmarks including UnKEBench, AKEW, and our new EditEverything dataset for long-form diverse-formatted knowledge. Additionally, AnyEdit serves as a plug-and-play framework, enabling current editing methods to update knowledge with arbitrary length and format, significantly advancing the scope and practicality of LLM knowledge editing.