mlsanityData to 7 Oct 2026

Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks

arXiv 2112.10017 ↗First posted 18 Dec 2021cs.LG cs.AI cs.CVcited by 80

Not on the map. Why?

Abstract (arXiv)

Existing research on continual learning of a sequence of tasks focused on dealing with catastrophic forgetting, where the tasks are assumed to be dissimilar and have little shared knowledge. Some work has also been done to transfer previously learned knowledge to the new task when the tasks are similar and have shared knowledge. To the best of our knowledge, no technique has been proposed to learn a sequence of mixed similar and dissimilar tasks that can deal with forgetting and also transfer knowledge forward and backward. This paper proposes such a technique to learn both types of tasks in the same network. For dissimilar tasks, the algorithm focuses on dealing with forgetting, and for similar tasks, the algorithm focuses on selectively transferring the knowledge learned from some similar previous tasks to improve the new task learning. Additionally, the algorithm automatically detects whether a new task is similar to any previous tasks. Empirical evaluation using sequences of mixed tasks demonstrates the effectiveness of the proposed model.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). Builds on 38 of its 60 references that are in the map. First posted on arXiv 18 Dec 2021.

Builds on

38 of 60 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2016Learning without Forgettingcited by 2,465
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  6. 2016Progressive Neural Networkscited by 1,332
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2017Continual Learning with Deep Generative Replaycited by 1,152
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  10. 2018Experience Replay for Continual Learningcited by 1,074
  11. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  12. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
26 more
  1. 2018LEAF: A Benchmark for Federated Settingscited by 780
  2. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  3. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  4. 2019Three scenarios for continual learningcited by 610
  5. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  6. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  7. 2017Variational Continual Learningcited by 335
  8. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  9. 2019Continual learning with hypernetworkscited by 267
  10. 2018Learning without Memorizingcited by 249
  11. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  12. 2019Meta-Learning Representations for Continual Learningcited by 189
  13. 2017FearNet: Brain-Inspired Model for Incremental Learningcited by 177
  14. 2019Uncertainty-based Continual Learning with Adaptive Regularizationcited by 144
  15. 2019A continual learning survey: Defying forgetting in classification taskscited by 118
  16. 2017Incremental Learning Through Deep Adaptationcited by 108
  17. 2018Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilizationcited by 104
  18. 2020iTAML: An Incremental Task-Agnostic Meta-learning Approachcited by 90
  19. 2017Deep Generative Dual Memory Network for Continual Learningcited by 59
  20. 2017Continual Learning in Generative Adversarial Netscited by 51
  21. 2019Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replaycited by 51
  22. 2019Policy Consolidation for Continual Reinforcement Learningcited by 37
  23. 2018Lifelong Learning for Sentiment Classificationcited by 21
  24. 2017A Strategy for an Uncompromising Incremental Learnercited by 9
  25. 2017Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networkscited by 9
  26. 2019Forward and Backward Knowledge Transfer for Sentiment Classificationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 19; 2023: 21; 2024: 19; 2025: 11; 2026: 10.

80 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  3. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  4. 2022A Theoretical Study on Solving Continual Learningcited by 78
  5. 2023Theory on Forgetting and Generalization of Continual Learningcited by 69
  6. 2022CLIP model is an Efficient Continual Learnercited by 64
  7. 2022Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfercited by 59
  8. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  9. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  10. 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 52
  11. 2023Introducing Language Guidance in Prompt-based Continual Learningcited by 49
  12. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
38 more
  1. 2022On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learningcited by 39
  2. 2023Parameter-Level Soft-Masking for Continual Learningcited by 39
  3. 2023Continual Learning on Dynamic Graphs via Parameter Isolationcited by 35
  4. 2023Learnability and Algorithm for Continual Learningcited by 28
  5. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  6. 2022A Multi-Head Model for Continual Learning via Out-of-Distribution Replaycited by 27
  7. 2024Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptationcited by 26
  8. 2023Prompt-augmented Temporal Point Process for Streaming Event Sequencecited by 24
  9. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  10. 2022AI Autonomy : Self-Initiated Open-World Continual Learning and Adaptationcited by 23
  11. 2023Adapting a Language Model While Preserving its General Knowledgecited by 23
  12. 2022Continual Learning Based on OOD Detection and Task Maskingcited by 21
  13. 2022Challenging Common Assumptions about Catastrophic Forgettingcited by 20
  14. 2024Disentangling and Mitigating the Impact of Task Similarity for Continual Learningcited by 19
  15. 2022A Theory for Knowledge Transfer in Continual Learningcited by 16
  16. 2023Learning to Prompt Knowledge Transfer for Open-World Continual Learningcited by 15
  17. 2024Empowering Large Language Model for Continual Video Question Answering with Collaborative Promptingcited by 14
  18. 2025Optimal Task Order for Continual Learning of Multiple Taskscited by 14
  19. 2022Lifelong Learning Metricscited by 13
  20. 2024Continual Learning in the Frequency Domaincited by 12
  21. 2023Task Difficulty Aware Parameter Allocation & Regularization for Lifelong Learningcited by 11
  22. 2025ProtoDepth: Unsupervised Continual Depth Completion with Prototypescited by 11
  23. 2023Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Taskscited by 10
  24. 2023Lifelong Sequence Generation with Dynamic Module Expansion and Adaptationcited by 10
  25. 2024Queryable Prototype Multiple Instance Learning with Vision-Language Models for Incremental Whole Slide Image Classificationcited by 10
  26. 2025Lightspeed Geometric Dataset Distance via Sliced Optimal Transportcited by 10
  27. 2024LW2G: Learning Whether to Grow for Prompt-based Continual Learningcited by 9
  28. 2022Theoretical Understanding of the Information Flow on Continual Learning Performancecited by 8
  29. 2024PECTP: Parameter-Efficient Cross-Task Prompts for Incremental Vision Transformercited by 8
  30. 2023Knowledge Restore and Transfer for Multi-label Class-Incremental Learningcited by 7
  31. 2023Lightweight Learner for Shared Knowledge Lifelong Learningcited by 7
  32. 2023Task Relation-aware Continual User Representation Learningcited by 7
  33. 2023Sub-network Discovery and Soft-masking for Continual Learning of Mixed Taskscited by 7
  34. 2024Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learningcited by 7
  35. 2022AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasetscited by 4
  36. 2024Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaborationcited by 4
  37. 2024COBRA: A Continual Learning Approach to Vision-Brain Understandingcited by 4
  38. 2025Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updatescited by 4

50 most cited of 80 All 80 (JSON)