# Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing

arXiv: https://arxiv.org/abs/1712.02719 · HTML: https://mlsanity.com/paper/1712.02719/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2017.

## Built on by (most cited first, 22 of 22)

- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Controllable Semantic Parsing via Retrieval Augmentation](https://mlsanity.com/paper/2110.08458/) (2021)
- [GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge](https://mlsanity.com/paper/2201.07705/) (2022)
- [Deep Bayesian Unsupervised Lifelong Learning](https://mlsanity.com/paper/2106.07035/) (2021)
- [Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning](https://mlsanity.com/paper/2309.01289/) (2023)
- [Cognitively Inspired Learning of Incremental Drifting Concepts](https://mlsanity.com/paper/2110.04662/) (2021)
- [Continual Unlearning for Foundational Text-to-Image Models without Generalization Erosion](https://mlsanity.com/paper/2503.13769/) (2025)
- [DaCapo: Accelerating Continuous Learning in Autonomous Systems for Video Analytics](https://mlsanity.com/paper/2403.14353/) (2024)
- [Incremental Multi-Target Domain Adaptation for Object Detection with Efficient Domain Transfer](https://mlsanity.com/paper/2104.06476/) (2021)
- [Rethinking Hard-Parameter Sharing in Multi-Domain Learning](https://mlsanity.com/paper/2107.11359/) (2021)
- [Addressing Client Drift in Federated Continual Learning with Adaptive Optimization](https://mlsanity.com/paper/2203.13321/) (2022)
- [Detecting and Learning the Unknown in Semantic Segmentation](https://mlsanity.com/paper/2202.08700/) (2022)
- [Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery](https://mlsanity.com/paper/2402.05860/) (2024)
- [CroMo-Mixup: Augmenting Cross-Model Representations for Continual Self-Supervised Learning](https://mlsanity.com/paper/2407.12188/) (2024)
- [Complexity-aware Adaptive Training and Inference for Edge-Cloud Distributed AI Systems](https://mlsanity.com/paper/2109.06440/) (2021)
- [Accretionary Learning with Deep Neural Networks](https://mlsanity.com/paper/2111.10857/) (2021)
- [Algorithm Design for Online Meta-Learning with Task Boundary Detection](https://mlsanity.com/paper/2302.00857/) (2023)
- [On The Relationship Between Continual Learning and Long-Tailed Recognition](https://mlsanity.com/paper/2306.13275/) (2023)
- [Configurable Holography: Towards Display and Scene Adaptation](https://mlsanity.com/paper/2405.01558/) (2024)
- [Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency](https://mlsanity.com/paper/2509.17695/) (2025)
- [Efficient Meta-Learning for Continual Learning with Taylor Expansion Approximation](https://mlsanity.com/paper/2210.00713/) (2022)
- [UnifiedNN: Efficient Neural Network Training on the Cloud](https://mlsanity.com/paper/2408.01331/) (2024)

## Abstract (arXiv)

Deep convolutional neural network (DCNN) based supervised learning is a widely practiced approach for large-scale image classification. However, retraining these large networks to accommodate new, previously unseen data demands high computational time and energy requirements. Also, previously seen training samples may not be available at the time of retraining. We propose an efficient training methodology and incrementally growing DCNN to learn new tasks while sharing part of the base network. Our proposed methodology is inspired by transfer learning techniques, although it does not forget previously learned tasks. An updated network for learning new set of classes is formed using previously learned convolutional layers (shared from initial part of base network) with addition of few newly added convolutional kernels included in the later layers of the network. We employed a `clone-and-branch' technique which allows the network to learn new tasks one after another without any performance loss in old tasks. We evaluated the proposed scheme on several recognition applications. The classification accuracy achieved by our approach is comparable to the regular incremental learning approach (where networks are updated with new training samples only, without any network sharing), while achieving energy efficiency, reduction in storage requirements, memory access and training time.