mlsanityData to 7 Oct 2026

Editing Factual Knowledge in Language Models

arXiv 2104.08164 ↗First posted 16 Apr 2021cs.CL cs.AI cs.LGcited by 548

Not on the map. Why?

Cited by 33 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts can be incorrectly induced or become obsolete over time. We present KnowledgeEditor, a method which can be used to edit this knowledge and, thus, fix 'bugs' or unexpected predictions without the need for expensive re-training or fine-tuning. Besides being computationally efficient, KnowledgeEditordoes not require any modifications in LM pre-training (e.g., the use of meta-learning). In our approach, we train a hyper-network with constrained optimization to modify a fact without affecting the rest of the knowledge; the trained hyper-network is then used to predict the weight update at test time. We show KnowledgeEditor's efficacy with two popular architectures and knowledge-intensive tasks: i) a BERT model fine-tuned for fact-checking, and ii) a sequence-to-sequence BART model for question answering. With our method, changing a prediction on the specific wording of a query tends to result in a consistent change in predictions also for its paraphrases. We show that this can be further encouraged by exploiting (e.g., automatically-generated) paraphrases during training. Interestingly, our hyper-network can be regarded as a 'probe' revealing which components need to be changed to manipulate factual knowledge; our analysis shows that the updates tend to be concentrated on a small subset of components. Source code available at https://github.com/nicola-decao/KnowledgeEditor

Key facts. Cited by 548 papers in the map (data through 7 Oct 2026). Builds on 39 of its 63 references that are in the map. First posted on arXiv 16 Apr 2021.

Builds on

39 of 63 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  9. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  11. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
27 more
  1. 2019Language Models as Knowledge Bases?cited by 1,922
  2. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  3. 2016HyperNetworkscited by 1,003
  4. 2019How Can We Know What Language Models Know?cited by 953
  5. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  6. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  7. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  8. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  9. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  10. 2020Autoregressive Entity Retrievalcited by 333
  11. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  12. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  13. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  14. 2020Modifying Memories in Transformer Modelscited by 222
  15. 2016Model-Agnostic Interpretability of Machine Learningcited by 219
  16. 2018Marian: Fast Neural Machine Translation in C++cited by 202
  17. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  18. 2020Editable Neural Networkscited by 154
  19. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  20. 2020Entities as Experts: Sparse Memory Access with Entity Supervisioncited by 100
  21. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 97
  22. 2019The emergence of number and syntax units in LSTM language modelscited by 92
  23. 2017Bayesian Hypernetworkscited by 64
  24. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  25. 2020Language Models as Fact Checkers?cited by 51
  26. 2021Multilingual Autoregressive Entity Linkingcited by 50
  27. 2020Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledgecited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 43; 2023: 107; 2024: 197; 2025: 118; 2026: 75.

548 papers.

Sort:

By area:

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2022Editing Models with Task Arithmeticcited by 1,076
  4. 2022Mass-Editing Memory in a Transformercited by 976
  5. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  6. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  7. 2022A Survey on In-context Learningcited by 717
  8. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  9. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  10. 2023Erasing Concepts from Diffusion Modelscited by 530
  11. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  12. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 507
38 more
  1. 2021Fast Model Editing at Scalecited by 491
  2. 2022Memory-Based Model Editing at Scalecited by 443
  3. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  4. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 376
  5. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  6. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  7. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  8. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  9. 2022Post-hoc Concept Bottleneck Modelscited by 289
  10. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  11. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  12. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  13. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  14. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  15. 2022Prompting GPT-3 To Be Reliablecited by 258
  16. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  17. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  18. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  19. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  20. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  21. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  22. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  23. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  24. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 196
  25. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  26. 2022Patching open-vocabulary models by interpolating weightscited by 187
  27. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  28. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  29. 2023PMET: Precise Model Editing in a Transformercited by 170
  30. 2022Large Language Models with Controllable Working Memorycited by 169
  31. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  32. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  33. 2022A Review on Language Models as Knowledge Basescited by 156
  34. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  35. 2021MoEfication: Transformer Feed-forward Layers are Mixtures of Expertscited by 149
  36. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  37. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  38. 2023Taken out of context: On measuring situational awareness in LLMscited by 131

50 most cited of 548 All 548 (JSON)