# Knowledge transfer in deep block-modular neural networks

arXiv: https://arxiv.org/abs/1908.08017 · HTML: https://mlsanity.com/paper/1908.08017/

## Key facts

- Cited by 14 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 14 of 14)

- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation](https://mlsanity.com/paper/2303.17051/) (2023)
- [Self-Composing Policies for Scalable Continual Reinforcement Learning](https://mlsanity.com/paper/2506.14811/) (2025)
- [Self-Activating Neural Ensembles for Continual Reinforcement Learning](https://mlsanity.com/paper/2301.00141/) (2022)
- [Jointly Exploring Client Drift and Catastrophic Forgetting in Dynamic Learning](https://mlsanity.com/paper/2309.00688/) (2023)
- [Directed Structural Adaptation to Overcome Statistical Conflicts and Enable Continual Learning](https://mlsanity.com/paper/2412.04190/) (2024)
- [Cold Start Streaming Learning for Deep Networks](https://mlsanity.com/paper/2211.04624/) (2022)
- [Modularity in Deep Learning: A Survey](https://mlsanity.com/paper/2310.01154/) (2023)
- [Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization](https://mlsanity.com/paper/2309.08546/) (2023)
- [Memory Efficient Adaptive Attention For Multiple Domain Learning](https://mlsanity.com/paper/2110.10969/) (2021)
- [CNT (Conditioning on Noisy Targets): A new Algorithm for Leveraging Top-Down Feedback](https://mlsanity.com/paper/2210.09505/) (2022)
- [GMM-IL: Image Classification using Incrementally Learnt, Independent Probabilistic Models for Small Sample Sizes](https://mlsanity.com/paper/2212.00572/) (2022)
- [Adaptive Growth: Real-time CNN Layer Expansion](https://mlsanity.com/paper/2309.03049/) (2023)
- [In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models](https://mlsanity.com/paper/2608.19514/) (2026)

## Abstract (arXiv)

Although deep neural networks (DNNs) have demonstrated impressive results during the last decade, they remain highly specialized tools, which are trained -- often from scratch -- to solve each particular task. The human brain, in contrast, significantly re-uses existing capacities when learning to solve new tasks. In the current study we explore a block-modular architecture for DNNs, which allows parts of the existing network to be re-used to solve a new task without a decrease in performance when solving the original task. We show that networks with such architectures can outperform networks trained from scratch, or perform comparably, while having to learn nearly 10 times fewer weights than the networks trained from scratch.