mlsanityData to 14 Oct 2026

Partial Network Cloning

arXiv 2303.10597 ↗First posted 19 Mar 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we study a novel task that enables partial knowledge transfer from pre-trained models, which we term as Partial Network Cloning (PNC). Unlike prior methods that update all or at least part of the parameters in the target network throughout the knowledge transfer process, PNC conducts partial parametric "cloning" from a source network and then injects the cloned module to the target, without modifying its parameters. Thanks to the transferred module, the target network is expected to gain additional functionality, such as inference on new classes; whenever needed, the cloned module can be readily removed from the target, with its original parameters and competence kept intact. Specifically, we introduce an innovative learning scheme that allows us to identify simultaneously the component to be cloned from the source and the position to be inserted within the target network, so as to ensure the optimal performance. Experimental results on several datasets demonstrate that, our method yields a significant improvement of 5% in accuracy and 50% in locality when compared with parameter-tuning based methods. Our code is available at https://github.com/JngwenYe/PNCloning.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 46 of its 61 references that are in the map. First posted on arXiv 19 Mar 2023.

Builds on

46 of 61 references.

References · 46

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  2. 2016Learning without Forgettingcited by 2,496
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  4. 2017Continual Learning with Deep Generative Replaycited by 1,158
  5. 2019Large Scale Incremental Learningcited by 732
  6. 2018End-to-End Incremental Learningcited by 597
  7. 2021Editing Factual Knowledge in Language Modelscited by 553
  8. 2021Fast Model Editing at Scalecited by 498
  9. 2022Memory-Based Model Editing at Scalecited by 447
  10. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  11. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  12. 2017Incremental Learning of Object Detectors without Catastrophic Forgettingcited by 208
34 more
  1. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  2. 2023DepGraph: Towards Any Structural Pruningcited by 203
  3. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  4. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  5. 2020Editable Neural Networkscited by 155
  6. 2020Distilling Knowledge from Graph Convolutional Networkscited by 129
  7. 2019Scalable and Order-robust Continual Learning with Additive Parameter Decompositioncited by 125
  8. 2019Functional Regularisation for Continual Learning with Gaussian Processescited by 123
  9. 2022Diffusion Probabilistic Model Made Slimcited by 121
  10. 2022Dataset Distillation via Factorizationcited by 115
  11. 2020Continual Learning in Low-rank Orthogonal Subspacescited by 99
  12. 2020Overcoming Catastrophic Forgetting in Graph Neural Networkscited by 95
  13. 2020Factorizable Graph Convolutional Networkscited by 79
  14. 2022Deep Model Reassemblycited by 78
  15. 2021Adaptive Multi-Teacher Multi-level Knowledge Distillationcited by 72
  16. 2022Factorizing Knowledge in Neural Networkscited by 59
  17. 2018MEAL: Multi-Model Ensemble via Adversarial Learningcited by 52
  18. 2021Provable Repair of Deep Neural Networkscited by 42
  19. 2021Co-advise: Cross Inductive Bias Distillationcited by 40
  20. 2020Data-Free Knowledge Amalgamation via Group-Stack Dual-GANcited by 32
  21. 2022Learning with Recoverable Forgettingcited by 30
  22. 2022Learning Graph Neural Networks for Image Style Transfercited by 29
  23. 2020Growing Efficient Deep Networks by Structured Continuous Sparsificationcited by 28
  24. 2021Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networkscited by 25
  25. 2022DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generationcited by 24
  26. 2020Online Ensemble Model Compression using Knowledge Distillationcited by 16
  27. 2021GROWN: GRow Only When Necessary for Continual Learningcited by 11
  28. 2020Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptationcited by 9
  29. 2021Safe Distillation Boxcited by 9
  30. 2022Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuningcited by 9
  31. 2017Knowledge Concentration: Learning 100K Object Classifiers in a Single CNNcited by 6
  32. 2019Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teacherscited by 6
  33. 2021FFNB: Forgetting-Free Neural Blocks for Deep Continual Visual Learningcited by 6
  34. 2019Autoencoder-Based Incremental Class Learning without Retraining on Old Datacited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4.

6 papers.

  1. 2023Distribution Shift Inversion for Out-of-Distribution Predictioncited by 23
  2. 2023Master: Meta Style Transformer for Controllable Zero-Shot and Few-Shot Artistic Style Transfercited by 9
  3. 2024Ungeneralizable Examplescited by 9
  4. 2024Neural Lineagecited by 9
  5. 2024StyDeSty: Min-Max Stylization and Destylization for Single Domain Generalizationcited by 5
  6. 2024Distilled Datamodel with Reverse Gradient Matchingcited by 3