mlsanityData to 7 Oct 2026

Mass-Editing Memory in a Transformer

arXiv 2210.07229 ↗First posted 13 Oct 2022cs.CL cs.LGcited by 976

Not on the map. Why?

Cited by 109 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations. We develop MEMIT, a method for directly updating a language model with many memories, demonstrating experimentally that it can scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B), exceeding prior work by orders of magnitude. Our code and data are at https://memit.baulab.info.

Key facts. Cited by 976 papers in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

32 of 59 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  11. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 590
20 more
  1. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  2. 2021Editing Factual Knowledge in Language Modelscited by 548
  3. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  4. 2021Fast Model Editing at Scalecited by 491
  5. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  6. 2022Memory-Based Model Editing at Scalecited by 443
  7. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  8. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  9. 2013Direct and Indirect Effectscited by 276
  10. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  11. 2020Modifying Memories in Transformer Modelscited by 222
  12. 2020How Context Affects Language Models' Factual Predictionscited by 181
  13. 2022A Review on Language Models as Knowledge Basescited by 156
  14. 2020Editable Neural Networkscited by 154
  15. 2022Analyzing Transformers in Embedding Spacecited by 131
  16. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  17. 2021Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefscited by 79
  18. 2021Temporal Effects on Pre-trained Models for Language Processing Taskscited by 46
  19. 2022Kformer: Knowledge Injection in Transformer Feed-Forward Layerscited by 37
  20. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 110; 2024: 258; 2025: 279; 2026: 326.

976 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  4. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  5. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  6. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  7. 2023Unified Concept Editing in Diffusion Modelscited by 377
  8. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  9. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  10. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  11. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  12. 2023Language Models Represent Space and Timecited by 306
38 more
  1. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  2. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  3. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  4. 2023Function Vectors in Large Language Modelscited by 292
  5. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  6. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  7. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  8. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  9. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  10. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  11. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  12. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  13. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  14. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  15. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  16. 2023PMET: Precise Model Editing in a Transformercited by 170
  17. 2022Large Language Models with Controllable Working Memorycited by 169
  18. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  19. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  20. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  21. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  22. 2023Taken out of context: On measuring situational awareness in LLMscited by 131
  23. 2025MemOS: A Memory OS for AI Systemcited by 129
  24. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
  25. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  26. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  27. 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 116
  28. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  29. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  30. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
  31. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 101
  32. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  33. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  34. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 92
  35. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  36. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  37. 2024BadEdit: Backdooring large language models by model editingcited by 84
  38. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 84

50 most cited of 976 All 976 (JSON)