mlsanityData to 14 Oct 2026

PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

arXiv 1711.05769 ↗First posted 15 Nov 2017cs.CVcited by 898

Not on the map. Why?

Cited by 33 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper presents a method for adding multiple tasks to a single deep neural network while avoiding catastrophic forgetting. Inspired by network pruning techniques, we exploit redundancies in large deep networks to free up parameters that can then be employed to learn new tasks. By performing iterative pruning and network re-training, we are able to sequentially "pack" multiple tasks into a single network while ensuring minimal drop in performance and minimal storage overhead. Unlike prior work that uses proxy losses to maintain accuracy on older tasks, we always optimize for the task at hand. We perform extensive experiments on a variety of network architectures and large-scale datasets, and observe much better robustness against catastrophic forgetting than prior work. In particular, we are able to add three fine-grained classification tasks to a single ImageNet-trained VGG-16 network and achieve accuracies close to those of separately trained networks for each task. Code available at https://github.com/arunmallya/packnet

Key facts. Cited by 898 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 100; 2022: 129; 2023: 199; 2024: 174; 2025: 150; 2026: 146.

898 papers.

Sort:

By area:

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  3. 2021Learning to Prompt for Continual Learningcited by 944
  4. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  5. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  6. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  7. 2021Gradient Projection Memory for Continual Learningcited by 348
  8. 2021Towards Open World Object Detectioncited by 297
  9. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 245
  10. 2021Co2L: Contrastive Continual Learningcited by 244
  11. 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 221
  12. 2023Class-Incremental Learning: A Surveycited by 209
38 more
  1. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  2. 2021Generalized Source-free Domain Adaptationcited by 175
  3. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  4. 2021Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learningcited by 141
  5. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  6. 2022Continual Learning and Private Unlearningcited by 130
  7. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  8. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 124
  9. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  10. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  11. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  12. 2021Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representationscited by 96
  13. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
  14. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  15. 2021Continual Learning for Text Classification with Information Disentanglement Based Regularizationcited by 86
  16. 2021Invariant Information Bottleneck for Domain Generalizationcited by 86
  17. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  18. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  19. 2023Modular Deep Learningcited by 83
  20. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  21. 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 81
  22. 2022How catastrophic can catastrophic forgetting be in linear regression?cited by 81
  23. 2022A Theoretical Study on Solving Continual Learningcited by 80
  24. 2021Rehearsal revealed: The limits and merits of revisiting samples in continual learningcited by 76
  25. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  26. 2021Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learningcited by 72
  27. 2021Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Datacited by 69
  28. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 69
  29. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 67
  30. 2023Lifelong Language Pretraining with Distribution-Specialized Expertscited by 66
  31. 2022CLIP model is an Efficient Continual Learnercited by 65
  32. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  33. 2024CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Modelcited by 61
  34. 2022Class-Incremental Learning with Strong Pre-trained Modelscited by 60
  35. 2021Few-Shot Lifelong Learningcited by 59
  36. 2022Continual evaluation for lifelong learning: Identifying the stability gapcited by 56
  37. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  38. 2022SparCL: Sparse Continual Learning on the Edgecited by 55

50 most cited of 898 All 898 (JSON)