mlsanityData to 7 Oct 2026

Memory-Based Model Editing at Scale

arXiv 2206.06520 ↗First posted 13 Jun 2022cs.AI cs.CLcited by 443

Not on the map. Why?

Cited by 39 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Even the largest neural networks make errors, and once-correct predictions can become invalid as the world changes. Model editors make local updates to the behavior of base (pre-trained) models to inject updated knowledge or correct undesirable behaviors. Existing model editors have shown promise, but also suffer from insufficient expressiveness: they struggle to accurately model an edit's intended scope (examples affected by the edit), leading to inaccurate predictions for test inputs loosely related to the edit, and they often fail altogether after many edits. As a higher-capacity alternative, we propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC), which stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed. To enable more rigorous evaluation of model editors, we introduce three challenging language model editing problems based on question answering, fact-checking, and dialogue generation. We find that only SERAC achieves high performance on all three problems, consistently outperforming existing approaches to model editing by a significant margin. Code, data, and additional project information will be made available at https://sites.google.com/view/serac-editing.

Key facts. Cited by 443 papers in the map (data through 7 Oct 2026). Builds on 41 of its 51 references that are in the map. First posted on arXiv 13 Jun 2022.

Builds on

41 of 51 references.

References · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  11. 2016Matching Networks for One Shot Learningcited by 2,246
  12. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
29 more
  1. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  2. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  4. 2018Experience Replay for Continual Learningcited by 1,074
  5. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  6. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  7. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  8. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  9. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  10. 2014Neural Turing Machinescited by 630
  11. 2020Recipes for building an open-domain chatbotcited by 593
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  13. 2021Editing Factual Knowledge in Language Modelscited by 548
  14. 2021Fast Model Editing at Scalecited by 491
  15. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  16. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  17. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  18. 2020Modifying Memories in Transformer Modelscited by 222
  19. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  20. 2020Editable Neural Networkscited by 154
  21. 2017Neural Episodic Controlcited by 148
  22. 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110
  23. 2018Transforming Question Answering Datasets Into Natural Language Inference Datasetscited by 99
  24. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 97
  25. 2021Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefscited by 79
  26. 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61
  27. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56
  28. 2021Hindsight: Posterior-guided training of retrievers for improved open-ended generationcited by 38
  29. 2019Meta-Neighborhoodscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 68; 2024: 162; 2025: 109; 2026: 96.

443 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2022Editing Models with Task Arithmeticcited by 1,076
  4. 2022Mass-Editing Memory in a Transformercited by 976
  5. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  6. 2022Galactica: A Large Language Model for Sciencecited by 736
  7. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  8. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  9. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  10. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  11. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  12. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
38 more
  1. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  2. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  3. 2022Prompting GPT-3 To Be Reliablecited by 258
  4. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  5. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  6. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  7. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  8. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  9. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  10. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  11. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 196
  12. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  13. 2023PMET: Precise Model Editing in a Transformercited by 170
  14. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  15. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  16. 2025MemOS: A Memory OS for AI Systemcited by 129
  17. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  18. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  19. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 104
  20. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 101
  21. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  22. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 92
  23. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  24. 2024BadEdit: Backdooring large language models by model editingcited by 84
  25. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  26. 2023MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRAcited by 80
  27. 2024From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMscited by 77
  28. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  29. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 74
  30. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  31. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  32. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  33. 2023Parameter Efficient Multi-task Model Fusion with Partial Linearizationcited by 64
  34. 2023Massive Editing for Large Language Models via Meta Learningcited by 62
  35. 2023Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutionscited by 59
  36. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  37. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  38. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56

50 most cited of 443 All 443 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models
  2. 2025Benchmarking and Rethinking Knowledge Editing for Large Language Models
  3. 2024The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse
  4. 2025Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning
  5. 2026The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
  6. 2026Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models
  7. 2024Spectral Editing of Activations for Large Language Model Alignment
  8. 2024Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models