# A Review of Modularization Techniques in Artificial Neural Networks

arXiv: https://arxiv.org/abs/1904.12770 · HTML: https://mlsanity.com/paper/1904.12770/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2019.

## Built on by (most cited first, 19 of 19)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Continual Learning via Local Module Composition](https://mlsanity.com/paper/2111.07736/) (2021)
- [Incremental Residual Concept Bottleneck Models](https://mlsanity.com/paper/2404.08978/) (2024)
- [MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks](https://mlsanity.com/paper/2309.14118/) (2023)
- [Clustering units in neural networks: upstream vs downstream information](https://mlsanity.com/paper/2203.11815/) (2022)
- [Interpretability Needs a New Paradigm](https://mlsanity.com/paper/2405.05386/) (2024)
- [Neural Network Module Decomposition and Recomposition](https://mlsanity.com/paper/2112.13208/) (2021)
- [Applications of Spiking Neural Networks in Visual Place Recognition](https://mlsanity.com/paper/2311.13186/) (2023)
- [Computer vision for liquid samples in hospitals and medical labs using hierarchical image segmentation and relations prediction](https://mlsanity.com/paper/2105.01456/) (2021)
- [Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience](https://mlsanity.com/paper/2408.12664/) (2024)
- [Towards interpretable quantum machine learning via single-photon quantum walks](https://mlsanity.com/paper/2301.13669/) (2023)
- [Understanding Multimodal Deep Neural Networks: A Concept Selection View](https://mlsanity.com/paper/2404.08964/) (2024)
- [Modularity in Deep Learning: A Survey](https://mlsanity.com/paper/2310.01154/) (2023)
- [Multi-Excitation Projective Simulation with a Many-Body Physics Inspired Inductive Bias](https://mlsanity.com/paper/2402.10192/) (2024)
- [Modular Growth of Hierarchical Networks: Efficient, General, and Robust Curriculum Learning](https://mlsanity.com/paper/2406.06262/) (2024)
- [Advancing Perception in Artificial Intelligence through Principles of Cognitive Science](https://mlsanity.com/paper/2310.08803/) (2023)
- [Towards Understanding the Link Between Modularity and Performance in Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/2205.06451/) (2022)
- [Modular Grammatical Evolution for the Generation of Artificial Neural Networks](https://mlsanity.com/paper/2208.02787/) (2022)
- [The Generalist Brain Module: Module Repetition in Neural Networks in Light of the Minicolumn Hypothesis](https://mlsanity.com/paper/2507.12473/) (2025)

## Abstract (arXiv)

Artificial neural networks (ANNs) have achieved significant success in tackling classical and modern machine learning problems. As learning problems grow in scale and complexity, and expand into multi-disciplinary territory, a more modular approach for scaling ANNs will be needed. Modular neural networks (MNNs) are neural networks that embody the concepts and principles of modularity. MNNs adopt a large number of different techniques for achieving modularization. Previous surveys of modularization techniques are relatively scarce in their systematic analysis of MNNs, focusing mostly on empirical comparisons and lacking an extensive taxonomical framework. In this review, we aim to establish a solid taxonomy that captures the essential properties and relationships of the different variants of MNNs. Based on an investigation of the different levels at which modularization techniques act, we attempt to provide a universal and systematic framework for theorists studying MNNs, also trying along the way to emphasise the strengths and weaknesses of different modularization approaches in order to highlight good practices for neural network practitioners.