mlsanityData to 7 Oct 2026

Learning without Forgetting

arXiv 1606.09282 ↗First posted 29 Jun 2016cs.CV cs.LG stat.MLcited by 2,465

Not on the map. Why?

Cited by 96 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.

Key facts. Cited by 2,465 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 293; 2022: 377; 2023: 432; 2024: 499; 2025: 461; 2026: 403.

2,465 papers.

Sort:

By area:

  1. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  2. 2021Learning to Prompt for Continual Learningcited by 938
  3. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  4. 2021Domain Generalization: A Surveycited by 760
  5. 2023ModelScope Text-to-Video Technical Reportcited by 651
  6. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  7. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  8. 2022Continual Test-Time Domain Adaptationcited by 520
  9. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  10. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  11. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  12. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
38 more
  1. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 299
  2. 2021Towards Open World Object Detectioncited by 297
  3. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 296
  4. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  5. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  6. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  7. 2021Co2L: Contrastive Continual Learningcited by 243
  8. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  9. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 228
  10. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  11. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 215
  12. 2026Self-Distillation Enables Continual Learningcited by 212
  13. 2023Class-Incremental Learning: A Surveycited by 209
  14. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  15. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 196
  16. 2023Progressive Prompts: Continual Learning for Language Modelscited by 195
  17. 2022Dataset Condensation via Efficient Synthetic-Data Parameterizationcited by 187
  18. 2022Patching open-vocabulary models by interpolating weightscited by 187
  19. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
  20. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  21. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  22. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  23. 2023ZipIt! Merging Models from Different Tasks without Trainingcited by 178
  24. 2021Generalized Source-free Domain Adaptationcited by 175
  25. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  26. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  27. 2021Self-Supervised Models are Continual Learnerscited by 149
  28. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  29. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 140
  30. 2022Knowledge Distillation with the Reused Teacher Classifiercited by 140
  31. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  32. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 138
  33. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  34. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  35. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  36. 2022Federated Class-Incremental Learningcited by 126
  37. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  38. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125

50 most cited of 2,465 All 2,465 (JSON)