# Decomposing Convolutional Neural Networks into Reusable and Replaceable Modules

arXiv: https://arxiv.org/abs/2110.07720 · HTML: https://mlsanity.com/paper/2110.07720/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression](https://mlsanity.com/paper/1707.06342/) (2017)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Learning to Reason: End-to-End Module Networks for Visual Question Answering](https://mlsanity.com/paper/1704.05526/) (2017)
- [Dense Classification and Implanting for Few-Shot Learning](https://mlsanity.com/paper/1903.05050/) (2019)
- [Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness](https://mlsanity.com/paper/2005.12379/) (2020)
- [Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline](https://mlsanity.com/paper/2106.06054/) (2021)
- [One-Shot Generalization in Deep Generative Models](https://mlsanity.com/paper/1603.05106/) (2016)
- [DeepLocalize: Fault Localization for Deep Neural Networks](https://mlsanity.com/paper/2103.03376/) (2021)
- [Few-Shot Learning with Localization in Realistic Settings](https://mlsanity.com/paper/1904.08502/) (2019)
- [Recursive Sketches for Modular Deep Learning](https://mlsanity.com/paper/1905.12730/) (2019)
- [HSD-CNN: Hierarchically self decomposing CNN architecture using class specific filter sensitivity analysis](https://mlsanity.com/paper/1811.04406/) (2018)

## Built on by (most cited first, 14 of 14)

- [Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision](https://mlsanity.com/paper/2303.07476/) (2023)
- [Patching Weak Convolutional Neural Network Models through Modularization and Composition](https://mlsanity.com/paper/2209.06116/) (2022)
- [Reusing Deep Neural Network Models through Model Re-engineering](https://mlsanity.com/paper/2304.00245/) (2023)
- [Manas: Mining Software Repositories to Assist AutoML](https://mlsanity.com/paper/2112.03395/) (2021)
- [Pruning for Feature-Preserving Circuits in CNNs](https://mlsanity.com/paper/2206.01627/) (2022)
- [Modularizing while Training: A New Paradigm for Modularizing DNN Models](https://mlsanity.com/paper/2306.09376/) (2023)
- [LUNA: A Model-Based Universal Analysis Framework for Large Language Models](https://mlsanity.com/paper/2310.14211/) (2023)
- [Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement](https://mlsanity.com/paper/2212.05970/) (2022)
- [Modularity in Deep Learning: A Survey](https://mlsanity.com/paper/2310.01154/) (2023)
- [DNN Modularization via Activation-Driven Training](https://mlsanity.com/paper/2411.01074/) (2024)
- [Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment](https://mlsanity.com/paper/2401.14628/) (2024)
- [NeMo: A Neuron-Level Modularizing-While-Training Approach for Decomposing DNN Models](https://mlsanity.com/paper/2508.11348/) (2025)
- [Speculative Automated Refactoring of Imperative Deep Learning Programs to Graph Execution](https://mlsanity.com/paper/2504.05424/) (2025)
- [An Empirical Framework for Evaluating Semantic Preservation Using Hugging Face](https://mlsanity.com/paper/2512.07983/) (2025)

## Abstract (arXiv)

Training from scratch is the most common way to build a Convolutional Neural Network (CNN) based model. What if we can build new CNN models by reusing parts from previously build CNN models? What if we can improve a CNN model by replacing (possibly faulty) parts with other parts? In both cases, instead of training, can we identify the part responsible for each output class (module) in the model(s) and reuse or replace only the desired output classes to build a model? Prior work has proposed decomposing dense-based networks into modules (one for each output class) to enable reusability and replaceability in various scenarios. However, this work is limited to the dense layers and based on the one-to-one relationship between the nodes in consecutive layers. Due to the shared architecture in the CNN model, prior work cannot be adapted directly. In this paper, we propose to decompose a CNN model used for image classification problems into modules for each output class. These modules can further be reused or replaced to build a new model. We have evaluated our approach with CIFAR-10, CIFAR-100, and ImageNet tiny datasets with three variations of ResNet models and found that enabling decomposition comes with a small cost (1.77% and 0.85% for top-1 and top-5 accuracy, respectively). Also, building a model by reusing or replacing modules can be done with a 2.3% and 0.5% average loss of accuracy. Furthermore, reusing and replacing these modules reduces CO2e emission by ~37 times compared to training the model from scratch.