mlsanityData to 14 Oct 2026

Sequence Transferability and Task Order Selection in Continual Learning

arXiv 2502.06544 ↗First posted 10 Feb 2025cs.LG cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

In continual learning, understanding the properties of task sequences and their relationships to model performance is important for developing advanced algorithms with better accuracy. However, efforts in this direction remain underdeveloped despite encouraging progress in methodology development. In this work, we investigate the impacts of sequence transferability on continual learning and propose two novel measures that capture the total transferability of a task sequence, either in the forward or backward direction. Based on the empirical properties of these measures, we then develop a new method for the task order selection problem in continual learning. Our method can be shown to offer a better performance than the conventional strategy of random task selection.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 46 of its 62 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

46 of 62 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  3. 2016Learning without Forgettingcited by 2,496
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  5. 2016Matching Networks for One Shot Learningcited by 2,255
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  8. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  9. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  10. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  11. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  12. 2019Large Scale Incremental Learningcited by 732
34 more
  1. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  2. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
  3. 2019Orthogonal Gradient Descent for Continual Learningcited by 369
  4. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  5. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
  6. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 172
  7. 2019Transferability and Hardness of Supervised Classification Taskscited by 127
  8. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 124
  9. 2020Optimal Continual Learning has Perfect Memory and is NP-hardcited by 87
  10. 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 81
  11. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 72
  12. 2023Theory on Forgetting and Generalization of Continual Learningcited by 72
  13. 2019Toward Understanding Catastrophic Forgetting in Continual Learningcited by 68
  14. 2021OTCE: A Transferability Metric for Cross-Domain Cross-Task Representationscited by 62
  15. 2022Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfercited by 59
  16. 2021Transferability Estimation using Bhattacharyya Class Separabilitycited by 54
  17. 2020Ranking Neural Checkpointscited by 51
  18. 2021Frustratingly Easy Transferability Estimationcited by 45
  19. 2024Make Continual Learning Stronger via C-Flatcited by 35
  20. 2024Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Modelscited by 34
  21. 2022The Effect of Task Ordering in Continual Learningcited by 31
  22. 2022How stable are Transferability Metrics evaluations?cited by 30
  23. 2019Continual Learning in Practicecited by 28
  24. 2021Formalizing the Generalization-Forgetting Trade-off in Continual Learningcited by 24
  25. 2023A Comprehensive Empirical Evaluation on Online Continual Learningcited by 24
  26. 2021On Anytime Learning at Macroscalecited by 20
  27. 2023Is forgetting less a good inductive bias for forward transfer?cited by 18
  28. 2023ETran: Energy-Based Transferability Estimationcited by 18
  29. 2022Continual Learning for Visual Search with Backward Consistent Feature Embeddingcited by 17
  30. 2022A Theory for Knowledge Transfer in Continual Learningcited by 16
  31. 2023Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentationcited by 11
  32. 2023Exploring Model Transferability through the Lens of Potential Energycited by 11
  33. 2023Fast and Accurate Transferability Measurement by Evaluating Intra-class Feature Variancecited by 9
  34. 2023Simple Transferability Estimation for Regression Taskscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Ordercited by 9
  2. 2026Rethinking Transfer in Continual Learning: A Replay-Based Realisationcited by 1
  3. 2025Sequencing to Mitigate Catastrophic Forgetting in Continual Learningcited by 0
  4. 2026Learning to Order: Task Sequencing as In-Context Optimizationcited by 0
  5. 2026Ready2Blend: From Natural-Language Instructions to Composable Alignment Promptscited by 0