mlsanityData to 7 Oct 2026

Examining Forgetting in Continual Pre-training of Aligned Large Language Models

arXiv 2401.03129 ↗First posted 6 Jan 2024cs.CLcited by 16

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have exhibited remarkable proficiency across various tasks. Given the potent applications of LLMs in numerous fields, there has been a surge in LLM development. In developing LLMs, a common practice involves continual pre-training on previously fine-tuned models. However, this can lead to catastrophic forgetting. In our work, we investigate the phenomenon of forgetting that occurs during continual pre-training on an existing fine-tuned LLM. We evaluate the impact of continuous pre-training on the fine-tuned LLM across various dimensions, including output format, knowledge, and reliability. Experiment results highlight the non-trivial challenge of addressing catastrophic forgetting during continual pre-training, especially the repetition issue.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 33 references that are in the map. First posted on arXiv 6 Jan 2024.

Builds on

25 of 33 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  12. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
13 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  3. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  4. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  5. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  6. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  7. 2016FastText.zip: Compressing text classification modelscited by 331
  8. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  9. 2023Continual Pre-training of Language Modelscited by 194
  10. 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 135
  11. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  12. 2022ELLE: Efficient Lifelong Pre-training for Emerging Datacited by 70
  13. 2019How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representationscited by 63

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 5.

16 papers.

Sort:

By area:

  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  2. 2024Personalized Large Language Modelscited by 44
  3. 2025The Future of Continual Learning in the Era of Foundation Models: Three Key Directionscited by 16
  4. 2025Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Modelscited by 13
  5. 2024Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directionscited by 10
  6. 2025Mapping Post-Training Forgetting in Language Models at Scalecited by 8
  7. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  8. 2026Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stagescited by 4
  9. 2025Full-Parameter Continual Pretraining of Gemma2: Insights into Fluency and Domain Knowledgecited by 3
  10. 2025Merging Language and Domain Specific Models: The Impact on Technical Vocabulary Acquisitioncited by 2
  11. 2025CSP: A Simulator For Multi-Agent Ranking Competitionscited by 1
  12. 2025Adaptive Budget Allocation for Orthogonal-Subspace Adapter Tuning in LLMs Continual Learningcited by 1
4 more
  1. 2026Racka: Efficient Hungarian LLM Adaptation on Academic Infrastructurecited by 1
  2. 2026When to Review: Spaced Repetition for Continual Pre-Training of Language Modelscited by 1
  3. 2026Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Trainingcited by 0
  4. 2026Made in Hungary: Comments on the performance of generative language modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training
  2. 2025Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks
  3. 2026Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
  4. 2025When Continue Learning Meets Multimodal Large Language Model: A Survey
  5. 2024Continual Learning for Large Language Models: A Survey
  6. 2026Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
  7. 2023TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
  8. 2024SwitchCIT: Switching for Continual Instruction Tuning