mlsanityData to 14 Oct 2026

InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions

arXiv 2403.11435 ↗First posted 18 Mar 2024cs.CLcited by 58

Not on the map. Why?

Abstract (arXiv)

Instruction tuning effectively optimizes Large Language Models (LLMs) for downstream tasks. Due to the changing environment in real-life applications, LLMs necessitate continual task-specific adaptation without catastrophic forgetting. Considering the heavy computational cost, replay-based Continual Learning (CL) methods are the simplest and most widely used for LLMs to address the forgetting issue. However, traditional replay-based methods do not fully utilize instructions to customize the replay strategy. In this work, we propose a novel paradigm called Instruction-based Continual Learning (InsCL). InsCL dynamically replays previous data based on task similarity, calculated by Wasserstein Distance with instructions. Moreover, we further introduce an Instruction Information Metric (InsInfo) to quantify the complexity and diversity of instructions. According to InsInfo, InsCL guides the replay process more inclined to high-quality data. We conduct extensive experiments over 16 tasks with different training orders, observing consistent performance improvements of InsCL. When all tasks have been trained, InsCL achieves performance gains of 3.0 Relative Gain compared with Random Replay, and 27.96 Relative Gain compared with No Replay.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 48 of its 64 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

48 of 64 references.

References · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  11. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
36 more
  1. 2016Progressive Neural Networkscited by 1,361
  2. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  3. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  4. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  5. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  6. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  7. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
  8. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  9. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 303
  10. 2019Episodic Memory in Lifelong Language Learningcited by 188
  11. 2019Class-incremental Learning via Deep Model Consolidationcited by 173
  12. 2019LAMOL: LAnguage MOdeling for Lifelong Language Learningcited by 160
  13. 2020Geometric Dataset Distances via Optimal Transportcited by 158
  14. 2022Fine-tuned Language Models are Continual Learnerscited by 156
  15. 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversificationcited by 150
  16. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 115
  17. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 114
  18. 2021LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5cited by 104
  19. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  20. 2020Continual Learning in Task-Oriented Dialogue Systemscited by 83
  21. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  22. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  23. 2022ELLE: Efficient Lifelong Pre-training for Emerging Datacited by 70
  24. 2020On the Transformer Growth for Progressive BERT Trainingcited by 49
  25. 2020Efficient Meta Lifelong-Learning with Limited Memorycited by 48
  26. 2020Continual Learning for Natural Language Generation in Task-oriented Dialog Systemscited by 47
  27. 2021Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Explorationcited by 44
  28. 2023Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curationcited by 38
  29. 2021Continual Learning of Knowledge Graph Embeddingscited by 35
  30. 2021A Survey on Optimal Transport for Machine Learning: Theory and Applicationscited by 35
  31. 2022Continual Sequence Generation with Adaptive Compositional Modulescited by 31
  32. 2022Wasserstein Task Embedding for Measuring Task Similaritiescited by 25
  33. 2023Specialist or Generalist? Instruction Tuning for Specific NLP Taskscited by 17
  34. 2021Lifelong Intent Detection via Multi-Strategy Rebalancingcited by 12
  35. 2022TIDF-DLPM: Term and Inverse Document Frequency based Data Leakage Prevention Modelcited by 4
  36. 2023Continual Learning via Manifold Expansion Replaycited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 18; 2025: 19; 2026: 21.

58 papers.

Sort:

By area:

  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  2. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 64
  3. 2024Recent Advances of Multimodal Continual Learning: A Comprehensive Surveycited by 39
  4. 2024ModalPrompt: Towards Efficient Multimodal Continual Instruction Tuning with Dual-Modality Guided Promptcited by 30
  5. 2025Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuningcited by 27
  6. 2025MorphMark: Flexible Adaptive Watermarking for Large Language Modelscited by 24
  7. 2025SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMscited by 24
  8. 2024Unlocking Continual Learning Abilities in Language Modelscited by 23
  9. 2024Self-Updatable Large Language Models by Integrating Context into Model Parameterscited by 21
  10. 2025Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuningcited by 16
  11. 2025Recurrent Knowledge Identification and Fusion for Language Model Continual Learningcited by 15
  12. 2024MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuningcited by 14
38 more
  1. 2024Towards LifeSpan Cognitive Systemscited by 13
  2. 2025Bisecle: Binding and Separation in Continual Learning for Video Language Understandingcited by 13
  3. 2025Continual Learning for Generative AI: From LLMs to MLLMs and Beyondcited by 12
  4. 2024SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Modelscited by 11
  5. 2024KIF: Knowledge Identification and Fusion for Language Model Continual Learningcited by 9
  6. 2025From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignmentcited by 9
  7. 2024Demystifying Language Model Forgetting with Low-rank Example Associationscited by 8
  8. 2025Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learningcited by 8
  9. 2024LoRTA: Low Rank Tensor Adaptation of Large Language Modelscited by 7
  10. 2024In-context Continual Learning Assisted by an External Continual Learnercited by 7
  11. 2025What Makes a Good Natural Language Prompt?cited by 7
  12. 2025Continually Adding New Languages to Multilingual Language Modelscited by 7
  13. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  14. 2024Boosting LLM Translation Skills without General Ability Loss via Rationale Distillationcited by 6
  15. 2025Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMscited by 6
  16. 2026Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stagescited by 6
  17. 2026Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learningcited by 6
  18. 2024Continual Learning Using Only Large Language Model Promptingcited by 5
  19. 2025Data-efficient LLM Fine-tuning for Code Generationcited by 5
  20. 2026Decomposing the Basic Abilities of Large Language Models: Mitigating Cross-Task Interference in Multi-Task Instruct-Tuningcited by 5
  21. 2025Self-Evolving LLMs via Continual Instruction Tuningcited by 4
  22. 2024In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Taskscited by 3
  23. 2024Unveiling and Addressing Pseudo Forgetting in Large Language Modelscited by 3
  24. 2026ELLA: Efficient Lifelong Learning for Adapters in Large Language Modelscited by 3
  25. 2026Dynamic Proxy-Mixing: Transferring Replay Controllers from Small to Large Models for Continual Instruction Tuningcited by 3
  26. 2025Mitigating Forgetting in Low Rank Adaptationcited by 2
  27. 2026Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuningcited by 2
  28. 2025CrossICL: Cross-Task In-Context Learning via Unsupervised Demonstration Transfercited by 1
  29. 2025Efficient Code LLM Training via Distribution-Consistent and Diversity-Aware Data Selectioncited by 1
  30. 2026Sparse Adapter Fusion for Continual Learning in NLPcited by 1
  31. 2026Make LLM Learn to Synthesize from Streaming Experiences through Feedbackcited by 1
  32. 2026Rethinking Transfer in Continual Learning: A Replay-Based Realisationcited by 1
  33. 2026Bilinear Optimization Divergence: Diagnosing Factor-Constrained LoRA Continual Learningcited by 1
  34. 2026Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Modelscited by 1
  35. 2025Enhancing Memory Recall in LLMs with Gauss-Tin: A Hybrid Instructional and Gaussian Replay Approachcited by 0
  36. 2025CDT: A Comprehensive Capability Framework for Large Language Models Across Cognition, Domain, and Taskcited by 0
  37. 2026From Abstract to Contextual: What LLMs Still Cannot Do in Mathematicscited by 0
  38. 2026Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFTcited by 0

50 most cited of 58 All 58 (JSON)