mlsanityData to 7 Oct 2026

Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines

arXiv 1810.12488 ↗First posted 30 Oct 2018cs.LG cs.AI cs.CVcited by 228

Not on the map. Why?

Abstract (arXiv)

Continual learning has received a great deal of attention recently with several approaches being proposed. However, evaluations involve a diverse set of scenarios making meaningful comparison difficult. This work provides a systematic categorization of the scenarios and evaluates them within a consistent framework including strong baselines and state-of-the-art methods. The results provide an understanding of the relative difficulty of the scenarios and that simple baselines (Adagrad, L2 regularization, and naive rehearsal strategies) can surprisingly achieve similar performance to current mainstream methods. We conclude with several suggestions for creating harder evaluation scenarios and future research directions. The code is available at https://github.com/GT-RIPL/Continual-Learning-Benchmark

Key facts. Cited by 228 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 52; 2022: 48; 2023: 41; 2024: 40; 2025: 29; 2026: 18.

228 papers.

Sort:

By area:

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  3. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  4. 2021Gradient Projection Memory for Continual Learningcited by 338
  5. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  6. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  7. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  8. 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
  9. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  10. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  11. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  12. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 92
38 more
  1. 2021Rehearsal revealed: The limits and merits of revisiting samples in continual learningcited by 76
  2. 2024Continual Learning and Catastrophic Forgettingcited by 73
  3. 2021Wide Neural Networks Forget Less Catastrophicallycited by 64
  4. 2022Continual evaluation for lifelong learning: Identifying the stability gapcited by 55
  5. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  6. 2022A Closer Look at Rehearsal-Free Continual Learningcited by 54
  7. 2022An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditionscited by 54
  8. 2021Layerwise Optimization by Gradient Decomposition for Continual Learningcited by 51
  9. 2021Efficient Feature Transformations for Discriminative and Generative Continual Learningcited by 50
  10. 2021Powerpropagation: A sparsity inducing weight reparameterisationcited by 49
  11. 2022Architecture Matters in Continual Learningcited by 46
  12. 2022Class-Incremental Learning for Action Recognition in Videoscited by 46
  13. 2022CoSCL: Cooperation of Small Continual Learners is Stronger than a Big Onecited by 43
  14. 2021Class-Incremental Learning with Generative Classifierscited by 39
  15. 2021Online Continual Learning For Visual Food Classificationcited by 39
  16. 2023TiC-CLIP: Continual Training of CLIP Modelscited by 38
  17. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
  18. 2022The Challenges of Continuous Self-Supervised Learningcited by 36
  19. 2021Continual Learning of Knowledge Graph Embeddingscited by 35
  20. 2023Continual Learning with Scaled Gradient Projectioncited by 33
  21. 2024Towards Efficient Replay in Federated Incremental Learningcited by 33
  22. 2021Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffercited by 30
  23. 2022vCLIMB: A Novel Video Class Incremental Learning Benchmarkcited by 28
  24. 2022FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulationscited by 27
  25. 2021An Investigation of Replay-based Approaches for Continual Learningcited by 26
  26. 2021Formalizing the Generalization-Forgetting Trade-off in Continual Learningcited by 24
  27. 2023Fixed Design Analysis of Regularization-Based Continual Learningcited by 24
  28. 2022Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketchescited by 23
  29. 2023Low-rank extended Kalman filtering for online learning of neural networks from streaming datacited by 23
  30. 2022Online Continual Learning with Contrastive Vision Transformercited by 21
  31. 2024Adaptive Memory Replay for Continual Learningcited by 21
  32. 2021Condensed Composite Memory Continual Learningcited by 20
  33. 2022ConStruct-VL: Data-Free Continual Structured VL Concepts Learningcited by 20
  34. 2023Long-Tailed Continual Learning For Visual Food Recognitioncited by 20
  35. 2023Improving Plasticity in Online Continual Learning via Collaborative Learningcited by 20
  36. 2021Regularization-based Continual Learning for Anomaly Detection in Discrete Manufacturingcited by 19
  37. 2021Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognitioncited by 19
  38. 2021Unsupervised Continual Learning Via Pseudo Labelscited by 18

50 most cited of 228 All 228 (JSON)