mlsanityData to 7 Oct 2026

Continual Learning for Large Language Models: A Survey

arXiv 2402.01364 ↗First posted 2 Feb 2024cs.CL cs.LGcited by 144

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. However, updates are necessary to endow LLMs with new skills and keep them up-to-date with rapidly evolving human knowledge. This paper surveys recent works on continual learning for LLMs. Due to the unique nature of LLMs, we catalog continue learning techniques in a novel multi-staged categorization scheme, involving continual pretraining, instruction tuning, and alignment. We contrast continual learning for LLMs with simpler adaptation methods used in smaller models, as well as with other enhancement strategies like retrieval-augmented generation and model editing. Moreover, informed by a discussion of benchmarks and evaluation, we identify several challenges and future work directions for this crucial task.

Key facts. Cited by 144 papers in the map (data through 7 Oct 2026). Builds on 60 of its 69 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

60 of 69 references.

References · 60

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  9. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  10. 2017Continual Learning with Deep Generative Replaycited by 1,152
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
38 more
  1. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  2. 2020Artificial Intelligence, Values and Alignmentcited by 443
  3. 2023Augmented Language Models: a Surveycited by 398
  4. 2023Llemma: An Open Language Model For Mathematicscited by 385
  5. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  6. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 296
  7. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  8. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  9. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  10. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  11. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 211
  12. 2023Progressive Prompts: Continual Learning for Language Modelscited by 195
  13. 2023Continual Pre-training of Language Modelscited by 194
  14. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  15. 2023Mitigating the Alignment Tax of RLHFcited by 181
  16. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  17. 2020Continual Lifelong Learning in Natural Language Processing: A Surveycited by 152
  18. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  19. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
  20. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 117
  21. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 115
  22. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  23. 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
  24. 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 82
  25. 2024AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planningcited by 81
  26. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  27. 2023TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Modelscited by 59
  28. 2023Continual Learning: Applications and the Road Forwardcited by 59
  29. 2023Adapting Large Language Models to Domains via Reading Comprehensioncited by 48
  30. 2023TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systemscited by 39
  31. 2024SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Modelscited by 37
  32. 2021Curriculum-Meta Learning for Order-Robust Continual Relation Extractioncited by 34
  33. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  34. 2023Efficient Continual Pre-training for Building Domain Specific Large Language Modelscited by 33
  35. 2023CITB: A Benchmark for Continual Instruction Tuningcited by 32
  36. 2022ConTinTin: Continual Learning from Task Instructionscited by 26
  37. 2020Drinking from a Firehose: Continual Learning with Web-scale Natural Languagecited by 24
  38. 2023Exploring Continual Learning for Code Generation Modelscited by 22

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 60; 2025: 51; 2026: 33.

144 papers.

Sort:

By area:

  1. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  2. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  3. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  4. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  5. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  6. 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 82
  7. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  8. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  9. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 65
  10. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 60
  11. 2024Synthetic continued pretrainingcited by 56
  12. 2024Investigating Continual Pretraining in Large Language Models: Insights and Implicationscited by 54
38 more
  1. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  2. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  3. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  4. 2024Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoningcited by 41
  5. 2024CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuningcited by 41
  6. 2024Recent Advances of Multimodal Continual Learning: A Comprehensive Surveycited by 39
  7. 2024Open-FinLLMs: Open Multimodal Large Language Models for Financial Applicationscited by 36
  8. 2025SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuningcited by 32
  9. 2024Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptationcited by 27
  10. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  11. 2025Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuningcited by 27
  12. 2024iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvementcited by 23
  13. 2025Primus: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Trainingcited by 23
  14. 2025Pitfalls in Evaluating Language Model Forecasterscited by 23
  15. 2025SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMscited by 23
  16. 2024Unlocking Continual Learning Abilities in Language Modelscited by 22
  17. 2024A Practitioner's Guide to Continual Multimodal Pretrainingcited by 22
  18. 2024Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Modelscited by 22
  19. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  20. 2024Self-Updatable Large Language Models by Integrating Context into Model Parameterscited by 21
  21. 2024Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Modelscited by 21
  22. 2025MLLM-CL: Continual Learning for Multimodal Large Language Modelscited by 21
  23. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  24. 2024HFT: Half Fine-Tuning for Large Language Modelscited by 18
  25. 2024Should We Really Edit Language Models? On the Evaluation of Edited Language Modelscited by 18
  26. 2024SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Modelscited by 16
  27. 2024Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilitiescited by 16
  28. 2025Multimodal Large Language Models for Medicine: A Comprehensive Surveycited by 16
  29. 2025The Future of Continual Learning in the Era of Foundation Models: Three Key Directionscited by 16
  30. 2024Learning to Poison Large Language Models for Downstream Manipulationcited by 15
  31. 2024Is Parameter Collision Hindering Continual Learning in LLMs?cited by 15
  32. 2024VersiCode: Towards Version-controllable Code Generationcited by 14
  33. 2024LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Modelscited by 14
  34. 2024Propulsion: Steering LLM with Tiny Fine-Tuningcited by 14
  35. 2024DELIFT: Data Efficient Language model Instruction Fine Tuningcited by 14
  36. 2025LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillationcited by 14
  37. 2025Lightweight Online Adaption for Time Series Foundation Model Forecastscited by 14
  38. 2024MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuningcited by 13

50 most cited of 144 All 144 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025When Continue Learning Meets Multimodal Large Language Model: A Survey
  2. 2024Examining Forgetting in Continual Pre-training of Aligned Large Language Models
  3. 2024Improving Multimodal Large Language Models Using Continual Learning
  4. 2024MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting
  5. 2026Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting
  6. 2025Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
  7. 2024Towards Practical Tool Usage for Continually Learning LLMs
  8. 2024A critical review of methods and challenges in large language models