mlsanityData to 14 Oct 2026

A Unified Framework for Model Editing

arXiv 2403.14236 ↗First posted 21 Mar 2024cs.LG cs.AI cs.CLcited by 57

Not on the map. Why?

Abstract (arXiv)

ROME and MEMIT are largely believed to be two different model editing algorithms, with the major difference between them being the ability to perform batched edits. In this paper, we unify these two algorithms under a single conceptual umbrella, optimizing for the same goal, which we call the preservation-memorization objective. ROME uses an equality constraint to optimize this objective to perform one edit at a time, whereas MEMIT employs a more flexible least-square constraint that allows for batched edits. We generalize ROME and enable batched editing with equality constraint in the form of EMMET - an Equality-constrained Mass Model Editing algorithm for Transformers, a new batched memory-editing algorithm. EMMET can perform batched-edits up to a batch-size of 10,000, with very similar performance to MEMIT across multiple dimensions. With the introduction of EMMET, we truly unify ROME and MEMIT and show that both algorithms are equivalent in terms of their optimization objective, their abilities (singular and batched editing), their model editing performance and their limitations.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

20 of 37 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  6. 2022Mass-Editing Memory in a Transformercited by 1,001
  7. 2018Neural Network Acceptability Judgmentscited by 958
  8. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  9. 2021Editing Factual Knowledge in Language Modelscited by 553
  10. 2021Fast Model Editing at Scalecited by 498
  11. 2022Memory-Based Model Editing at Scalecited by 447
  12. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
8 more
  1. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  2. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  3. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
  4. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  5. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  6. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 93
  7. 2023Massive Editing for Large Language Models via Meta Learningcited by 63
  8. 2024Rebuilding ROME : Resolving Model Collapse during Sequential Model Editingcited by 50

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 23; 2026: 20.

57 papers.

Sort:

By area:

  1. 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 37
  2. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 34
  3. 2024Perturbation-Restrained Sequential Model Editingcited by 28
  4. 2025The Mirage of Model Editing: Revisiting Evaluation in the Wildcited by 27
  5. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  6. 2024Avoiding Copyright Infringement via Large Language Model Unlearningcited by 24
  7. 2024Neuron-Level Sequential Editing for Large Language Modelscited by 20
  8. 2024Representation Shattering in Transformers: A Synthetic Study with Knowledge Editingcited by 19
  9. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queriescited by 16
  10. 2024Memorization in deep learning: A surveycited by 15
  11. 2025Position: Editing Large Language Models Poses Serious Safety Riskscited by 15
  12. 2024Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Modelscited by 12
38 more
  1. 2025Lifelong Knowledge Editing requires Better Regularizationcited by 10
  2. 2025UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Modelscited by 9
  3. 2024Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3cited by 8
  4. 2024Concept-ROT: Poisoning Concepts in Large Language Models with Model Editingcited by 8
  5. 2025Context-Robust Knowledge Editing for Language Modelscited by 8
  6. 2025Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundationcited by 7
  7. 2025MEMIT-Merge: Addressing MEMIT's Key-Value Conflicts in Same-Subject Batch Editing for LLMscited by 6
  8. 2025GeoEdit: Geometric Knowledge Editing for Large Language Modelscited by 6
  9. 2025MindBridge: Scalable and Cross-Model Knowledge Editing via Memory-Augmented Modalitycited by 6
  10. 2025Disentangling Knowledge Representations for Large Language Model Editingcited by 6
  11. 2025First is Not Really Better Than Last: Evaluating Layer Choice and Aggregation Strategies in Language Model Data Influence Estimationcited by 5
  12. 2026CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editingcited by 5
  13. 2024Resolving Lexical Bias in Model Editingcited by 4
  14. 2024BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Modelscited by 4
  15. 2025EditMark: Watermarking Large Language Models based on Model Editingcited by 4
  16. 2026Are We Evaluating the Edit Locality of LLM Model Editing Properly?cited by 4
  17. 2025Efficient Knowledge Editing via Minimal Precomputationcited by 3
  18. 2025ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recallcited by 3
  19. 2025NAACL2025 Tutorial: Adaptation of Large Language Modelscited by 2
  20. 2025InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editingcited by 2
  21. 2026KUDA: Knowledge Unlearning by Deviating Representation for Large Language Modelscited by 2
  22. 2026CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editingcited by 2
  23. 2026From Backward Spreading to Forward Replay: Revisiting Target Construction in LLM Parameter Editingcited by 2
  24. 2025Norm Growth and Stability Challenges in Localized Sequential Knowledge Editingcited by 1
  25. 2025Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model Editingcited by 1
  26. 2025Koopman Autoencoders Learn Neural Representation Dynamicscited by 1
  27. 2025AUTOLAW: Enhancing Legal Compliance in Large Language Models via Case Law Generation and Jury-Inspired Deliberationcited by 1
  28. 2025Knowledge Fusion via Bidirectional Information Aggregationcited by 1
  29. 2025Patching LLM Like Software: A Lightweight Method for Improving Safety Policy in Large Language Modelscited by 1
  30. 2026Computational Compliance for AI Regulation: Blueprint for a New Research Domaincited by 1
  31. 2026Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Modelscited by 1
  32. 2026Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignmentcited by 1
  33. 2026Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysiscited by 1
  34. 2026BetaEdit: Null-Space Constrained Sequential Model Editingcited by 1
  35. 2026Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updatescited by 1
  36. 2026From Fact Overwriting to Knowledge Evolution: Causal Editing via On-Policy Self-Distillationcited by 1
  37. 2026Knowledge Editing for Masked Diffusion Language Modelscited by 1
  38. 2026Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMscited by 1

50 most cited of 57 All 57 (JSON)