mlsanityData to 7 Oct 2026

Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild

arXiv 1903.12648 ↗First posted 29 Mar 2019cs.CV cs.LG stat.MLcited by 111

Not on the map. Why?

Abstract (arXiv)

Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: the performance on previous tasks drastically degrades when learning a new task. To alleviate this effect, we propose to leverage a large stream of unlabeled data easily obtainable in the wild. In particular, we design a novel class-incremental learning scheme with (a) a new distillation loss, termed global distillation, (b) a learning strategy to avoid overfitting to the most recent task, and (c) a confidence-based sampling method to effectively leverage unlabeled external data. Our experimental results on various datasets, including CIFAR and ImageNet, demonstrate the superiority of the proposed methods over prior methods, particularly when a stream of unlabeled data is accessible: our method shows up to 15.8% higher accuracy and 46.5% less forgetting compared to the state-of-the-art method. The code is available at https://github.com/kibok90/iccv2019-inc.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 25; 2023: 29; 2024: 18; 2025: 12; 2026: 2.

111 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  3. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  4. 2023Class-Incremental Learning: A Surveycited by 209
  5. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 140
  6. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  7. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  8. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  9. 2022A Theoretical Study on Solving Continual Learningcited by 78
  10. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  11. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  12. 2025SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learningcited by 60
38 more
  1. 2022R-DFCIL: Relation-Guided Representation Learning for Data-Free Class Incremental Learningcited by 59
  2. 2021Essentials for Class Incremental Learningcited by 57
  3. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  4. 2022A Closer Look at Rehearsal-Free Continual Learningcited by 54
  5. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  6. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  7. 2021Layerwise Optimization by Gradient Decomposition for Continual Learningcited by 51
  8. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
  9. 2021Online Continual Learning For Visual Food Classificationcited by 39
  10. 2021Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connectorcited by 34
  11. 2023A Survey on Knowledge Editing of Neural Networkscited by 34
  12. 2021Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgettingcited by 33
  13. 2023Audio-Visual Class-Incremental Learningcited by 33
  14. 2023Class-Incremental Learning using Diffusion Model for Distillation and Replaycited by 31
  15. 2021Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffercited by 30
  16. 2022Balancing Stability and Plasticity through Advanced Null Space in Continual Learningcited by 29
  17. 2023Learnability and Algorithm for Continual Learningcited by 28
  18. 2022A Multi-Head Model for Continual Learning via Out-of-Distribution Replaycited by 27
  19. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  20. 2023Open-World Continual Learning: Unifying Novelty Detection and Continual Learningcited by 22
  21. 2022Continual Learning Based on OOD Detection and Task Maskingcited by 21
  22. 2021Prototype-Guided Memory Replay for Continual Learningcited by 19
  23. 2022Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Datacited by 19
  24. 2025Continual Multimodal Contrastive Learningcited by 19
  25. 2021Unsupervised Continual Learning Via Pseudo Labelscited by 18
  26. 2021On the importance of cross-task features for class-incremental learningcited by 18
  27. 2022How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysiscited by 18
  28. 2023Fast Trainable Projection for Robust Fine-Tuningcited by 18
  29. 2021Improving Vision Transformers for Incremental Learningcited by 17
  30. 2023Class Incremental Learning via Likelihood Ratio Based Task Predictioncited by 17
  31. 2023Label-Efficient Learning in Agriculture: A Comprehensive Reviewcited by 16
  32. 2023Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learningcited by 16
  33. 2023Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also Howcited by 16
  34. 2025Upweighting Easy Samples in Fine-Tuning Mitigates Forgettingcited by 16
  35. 2021Online Continual Learning Via Candidates Votingcited by 15
  36. 2022Class-Incremental Learning with Cross-Space Clustering and Controlled Transfercited by 15
  37. 2021Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Networkcited by 14
  38. 2022Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributionscited by 14

50 most cited of 111 All 111 (JSON)