mlsanityData to 14 Oct 2026

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

arXiv 1904.00310 ↗First posted 31 Mar 2019cs.LG cs.CVcited by 302

Not on the map. Why?

Abstract (arXiv)

Addressing catastrophic forgetting is one of the key challenges in continual learning where machine learning systems are trained with sequential or streaming tasks. Despite recent remarkable progress in state-of-the-art deep learning, deep neural networks (DNNs) are still plagued with the catastrophic forgetting problem. This paper presents a conceptually simple yet general and effective framework for handling catastrophic forgetting in continual learning with DNNs. The proposed method consists of two components: a neural structure optimization component and a parameter learning and/or fine-tuning component. By separating the explicit neural structure learning and the parameter estimation, not only is the proposed method capable of evolving neural structures in an intuitively meaningful way, but also shows strong capabilities of alleviating catastrophic forgetting in experiments. Furthermore, the proposed method outperforms all other baselines on the permuted MNIST dataset, the split CIFAR100 dataset and the Visual Domain Decathlon dataset in continual learning setting.

Key facts. Cited by 302 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 43; 2022: 54; 2023: 58; 2024: 67; 2025: 49; 2026: 31.

302 papers.

Sort:

By area:

  1. 2021Learning to Prompt for Continual Learningcited by 944
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  3. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  4. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  5. 2021Gradient Projection Memory for Continual Learningcited by 348
  6. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  7. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  8. 2023Class-Incremental Learning: A Surveycited by 209
  9. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
  10. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 143
  11. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 141
  12. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 125
38 more
  1. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 117
  2. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  3. 2021Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representationscited by 96
  4. 2021Representational Continuity for Unsupervised Continual Learningcited by 95
  5. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 90
  6. 2022TRGP: Trust Region Gradient Projection for Continual Learningcited by 82
  7. 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 81
  8. 2022Probing Representation Forgetting in Supervised and Unsupervised Continual Learningcited by 76
  9. 2021Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learningcited by 72
  10. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 69
  11. 2022CLIP model is an Efficient Continual Learnercited by 65
  12. 2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
  13. 2021Wide Neural Networks Forget Less Catastrophicallycited by 64
  14. 2022Class-Incremental Learning with Strong Pre-trained Modelscited by 60
  15. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  16. 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 54
  17. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  18. 2021Firefly Neural Architecture Descent: a General Approach for Growing Neural Networkscited by 49
  19. 2023Introducing Language Guidance in Prompt-based Continual Learningcited by 49
  20. 2021Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mappingcited by 48
  21. 2022Architecture Matters in Continual Learningcited by 46
  22. 2023AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learningcited by 46
  23. 2022Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interferencecited by 44
  24. 2022Continual Normalization: Rethinking Batch Normalization for Online Continual Learningcited by 40
  25. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 40
  26. 2021DualNet: Continual Learning, Fast and Slowcited by 39
  27. 2021Continually Learning Self-Supervised Representations with Projected Functional Regularizationcited by 38
  28. 2022On the Soft-Subnetwork for Few-shot Class Incremental Learningcited by 37
  29. 2022The Challenges of Continuous Self-Supervised Learningcited by 36
  30. 2023Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learningcited by 35
  31. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 35
  32. 2021Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connectorcited by 34
  33. 2024Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Modelscited by 34
  34. 2023Audio-Visual Class-Incremental Learningcited by 33
  35. 2021Continual Learning via Bit-Level Information Preservingcited by 32
  36. 2022Continual Learning with Recursive Gradient Optimizationcited by 32
  37. 2022Continual Sequence Generation with Adaptive Compositional Modulescited by 31
  38. 2022KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Predictioncited by 31

50 most cited of 302 All 302 (JSON)