mlsanityData to 7 Oct 2026

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

arXiv 2308.08747 ↗First posted 17 Aug 2023cs.CLcited by 622

Not on the map. Why?

Cited by 46 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Catastrophic forgetting (CF) is a phenomenon that occurs in machine learning when a model forgets previously learned information while acquiring new knowledge for achieving a satisfactory performance in downstream tasks. As large language models (LLMs) have demonstrated remarkable performance, it is intriguing to investigate whether CF exists during the continual instruction tuning of LLMs. This study empirically evaluates the forgetting phenomenon in LLMs' knowledge during continual instruction tuning from the perspectives of domain knowledge, reasoning, and reading comprehension. The experiments reveal that catastrophic forgetting is generally observed in LLMs ranging from 1b to 7b parameters. Surprisingly, as the model scale increases, the severity of forgetting intensifies in such a model sale range which may result from the much significant initial performance in the larger LLM. Comparing the decoder-only model BLOOMZ with the encoder-decoder model mT0, BLOOMZ exhibits less forgetting and retains more knowledge. Interestingly, we also observe that LLMs can mitigate language biases, such as gender bias, during continual fine-tuning. Furthermore, our findings indicate that general instruction tuning can help alleviate the forgetting phenomenon in LLMs during subsequent fine-tuning.

Key facts. Cited by 622 papers in the map (data through 7 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 17 Aug 2023.

Builds on

32 of 48 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2016Learning without Forgettingcited by 2,465
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
20 more
  1. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  2. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  3. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  4. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  5. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  6. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  7. 2019ELI5: Long Form Question Answeringcited by 567
  8. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  9. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  10. 2021MetaICL: Learning to Learn In Contextcited by 450
  11. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  12. 2020MuTual: A Dataset for Multi-Turn Dialogue Reasoningcited by 114
  13. 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
  14. 2020ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformationscited by 101
  15. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  16. 2022Probing Representation Forgetting in Supervised and Unsupervised Continual Learningcited by 74
  17. 2022Continual Training of Language Models for Few-Shot Learningcited by 33
  18. 2022AdaPrompt: Adaptive Model Training for Prompt-based NLPcited by 30
  19. 2023Investigating Forgetting in Pre-Trained Representations Through Continual Learningcited by 19
  20. 2023Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 27; 2024: 178; 2025: 199; 2026: 218.

622 papers.

Sort:

By area:

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  4. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  6. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  7. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  8. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  9. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  10. 2024Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performancecited by 179
  11. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  12. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
38 more
  1. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  2. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agentscited by 156
  3. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
  4. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  5. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  6. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  7. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Tokencited by 123
  8. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 114
  9. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  10. 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Modelcited by 112
  11. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
  12. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 90
  13. 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Modelscited by 90
  14. 2024SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agentscited by 89
  15. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  16. 2024BadEdit: Backdooring large language models by model editingcited by 84
  17. 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 82
  18. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  19. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  20. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 68
  21. 2024Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigationcited by 66
  22. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  23. 2025Are Reasoning Models More Prone to Hallucination?cited by 62
  24. 2024CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Modelcited by 60
  25. 2023TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Modelscited by 59
  26. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  27. 2024KG-RAG: Bridging the Gap Between Knowledge and Creativitycited by 55
  28. 2024Investigating Continual Pretraining in Large Language Models: Insights and Implicationscited by 54
  29. 2024InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructionscited by 54
  30. 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Promptscited by 53
  31. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languagescited by 52
  32. 2023Continual Instruction Tuning for Large Multimodal Modelscited by 49
  33. 2025The Best Instruction-Tuning Data are Those That Fitcited by 48
  34. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  35. 2023LM-Cocktail: Resilient Tuning of Language Models via Model Mergingcited by 44
  36. 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agentscited by 44
  37. 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessmentcited by 43
  38. 2026Fast and Effective On-policy Distillation from Reasoning Prefixescited by 43

50 most cited of 622 All 622 (JSON)