# Structure Learning for Neural Module Networks

arXiv: https://arxiv.org/abs/1905.11532 · HTML: https://mlsanity.com/paper/1905.11532/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 4 of 4)

- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [Personalized Federated Learning via Heterogeneous Modular Networks](https://mlsanity.com/paper/2210.14830/) (2022)
- [Modular Machine Learning: An Indispensable Path towards New-Generation Large Language Models](https://mlsanity.com/paper/2504.20020/) (2025)
- [Compositional Models: Multi-Task Learning and Knowledge Transfer with Modular Networks](https://mlsanity.com/paper/2107.10963/) (2021)

## Abstract (arXiv)

Neural Module Networks, originally proposed for the task of visual question answering, are a class of neural network architectures that involve human-specified neural modules, each designed for a specific form of reasoning. In current formulations of such networks only the parameters of the neural modules and/or the order of their execution is learned. In this work, we further expand this approach and also learn the underlying internal structure of modules in terms of the ordering and combination of simple and elementary arithmetic operators. Our results show that one is indeed able to simultaneously learn both internal module structure and module sequencing without extra supervisory signals for module execution sequencing. With this approach, we report performance comparable to models using hand-designed modules.