mlsanityData to 7 Oct 2026

Towards Robust Evaluations of Continual Learning

arXiv 1805.09733 ↗First posted 24 May 2018stat.ML cs.LGcited by 148

Not on the map. Why?

Abstract (arXiv)

Experiments used in current continual learning research do not faithfully assess fundamental challenges of learning continually. Instead of assessing performance on challenging and representative experiment designs, recent research has focused on increased dataset difficulty, while still using flawed experiment set-ups. We examine standard evaluations and show why these evaluations make some continual learning approaches look better than they are. We introduce desiderata for continual learning evaluations and explain why their absence creates misleading comparisons. Based on our desiderata we then propose new experiment designs which we demonstrate with various continual learning approaches and datasets. Our analysis calls for a reprioritization of research effort by the community.

Key facts. Cited by 148 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 46; 2022: 29; 2023: 35; 2024: 24; 2025: 8; 2026: 6.

148 papers.

Sort:

By area:

  1. 2021Learning to Prompt for Continual Learningcited by 938
  2. 2021Gradient Projection Memory for Continual Learningcited by 338
  3. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  4. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  5. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 215
  6. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 121
  7. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  8. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 110
  9. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 92
  10. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 86
  11. 2021Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Datacited by 67
  12. 2021Continual Learning via Local Module Compositioncited by 61
38 more
  1. 2021Model Zoo: A Growing "Brain" That Learns Continuallycited by 53
  2. 2021Posterior Meta-Replay for Continual Learningcited by 45
  3. 2022Transfer without Forgettingcited by 44
  4. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 42
  5. 2021Class-Incremental Learning with Generative Classifierscited by 39
  6. 2022On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learningcited by 39
  7. 2021Continual Learning for Blind Image Quality Assessmentcited by 36
  8. 2021Distilled Replay: Overcoming Forgetting through Synthetic Samplescited by 36
  9. 2021Continual Learning for Recurrent Neural Networks: an Empirical Evaluationcited by 34
  10. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 31
  11. 2022The Effect of Task Ordering in Continual Learningcited by 30
  12. 2023Online Continual Learning Without the Storage Constraintcited by 29
  13. 2021CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparabilitycited by 28
  14. 2022Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regimecited by 28
  15. 2023Continual Learning via Sequential Function-Space Variational Inferencecited by 28
  16. 2021Continual Learning on Noisy Data Streams via Self-Purified Replaycited by 27
  17. 2023Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learningcited by 27
  18. 2024The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Modelcited by 26
  19. 2024Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptationcited by 26
  20. 2021Continual Semi-Supervised Learning through Contrastive Interpolation Consistencycited by 25
  21. 2023BiRT: Bio-inspired Replay in Vision Transformers for Continual Learningcited by 25
  22. 2023Few-shot Class-incremental Learning for Classification and Object Detection: A Surveycited by 25
  23. 2024CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Modelscited by 25
  24. 2022Sylph: A Hypernetwork Framework for Incremental Few-shot Object Detectioncited by 23
  25. 2022Quantum Multi-Agent Meta Reinforcement Learningcited by 23
  26. 2021CLeaR: An Adaptive Continual Learning Framework for Regression Taskscited by 21
  27. 2021Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysiscited by 21
  28. 2021How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolationcited by 19
  29. 2021Sequoia: A Software Framework to Unify Continual Learning Researchcited by 19
  30. 2022Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activationscited by 18
  31. 2021Continual Learning in Deep Networks: an Analysis of the Last Layercited by 17
  32. 2024CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learningcited by 17
  33. 2022Sparse Coding in a Dual Memory System for Lifelong Learningcited by 16
  34. 2021Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgettingcited by 14
  35. 2022Consistency is the key to further mitigating catastrophic forgetting in continual learningcited by 14
  36. 2024Optimal Protocols for Continual Learning via Statistical Physics and Control Theorycited by 14
  37. 2022Lifelong Learning Metricscited by 13
  38. 2023A Domain-Agnostic Approach for Characterization of Lifelong Learning Systemscited by 13

50 most cited of 148 All 148 (JSON)