mlsanityData to 14 Oct 2026

Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models

arXiv 2305.09144 ↗First posted 16 May 2023cs.CL cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Memory is one of the most essential cognitive functions serving as a repository of world knowledge and episodes of activities. In recent years, large-scale pre-trained language models have shown remarkable memorizing ability. On the contrary, vanilla neural networks without pre-training have been long observed suffering from the catastrophic forgetting problem. To investigate such a retentive-forgetful contradiction and understand the memory mechanism of language models, we conduct thorough experiments by controlling the target knowledge types, the learning strategies and the learning schedules. We find that: 1) Vanilla language models are forgetful; 2) Pre-training leads to retentive language models; 3) Knowledge relevance and diversification significantly influence the memory formation. These conclusions are useful for understanding the abilities of pre-trained language models and shed light on designing and evaluating new learning and inference algorithms of language models.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 31 of its 75 references that are in the map. First posted on arXiv 16 May 2023.

Builds on

31 of 75 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2016Learning without Forgettingcited by 2,496
  6. 2020Extracting Training Data from Large Language Modelscited by 2,189
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  9. 2019How Can We Know What Language Models Know?cited by 959
  10. 2017A Survey on Multi-Task Learningcited by 840
  11. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  12. 2022Quantifying Memorization Across Neural Language Modelscited by 763
19 more
  1. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  2. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  3. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
  4. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  5. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  6. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  7. 2021Towards Continual Knowledge Learning of Language Modelscited by 174
  8. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  9. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  10. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  11. 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 120
  12. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  13. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  14. 2019Evaluating Commonsense in Pre-trained Language Modelscited by 95
  15. 2019Do Neural Language Representations Learn Physical Commonsense?cited by 62
  16. 2019A comprehensive, application-oriented study of catastrophic forgetting in DNNscited by 35
  17. 2021Memorisation versus Generalisation in Pre-trained Language Modelscited by 32
  18. 2022Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal Viewcited by 31
  19. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 8; 2025: 9.

18 papers.

Sort:

By area:

  1. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  2. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  3. 2024Continual Learning with Pre-Trained Models: A Surveycited by 154
  4. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  5. 2024How Much Can We Forget about Data Contamination?cited by 14
  6. 2024StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluationcited by 9
  7. 2025Revealing and Mitigating Over-Attention in Knowledge Editingcited by 8
  8. 2025Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Modelscited by 8
  9. 2025Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Groupingcited by 7
  10. 2025Persistent Backdoor Attacks under Continual Fine-Tuning of LLMscited by 7
  11. 2025Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretrainingcited by 6
  12. 2025Train Once, Answer All: Many Pretraining Experiments for the Cost of Onecited by 6
6 more
  1. 2025Do Your Best and Get Enough Rest for Continual Learningcited by 5
  2. 2024Derivational Morphology Reveals Analogical Generalization in Large Language Modelscited by 4
  3. 2024An Efficient Replay for Class-Incremental Learning with Pre-trained Modelscited by 2
  4. 2025The Lie of the Average: How Class Incremental Learning Evaluation Deceives You?cited by 2
  5. 2025Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigmscited by 1
  6. 2024Realistic Continual Learning Approach using Pre-trained Modelscited by 0