mlsanityData to 7 Oct 2026

How Modular Should Neural Module Networks Be for Systematic Generalization?

arXiv 2106.08170 ↗First posted 15 Jun 2021cs.LG cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Neural Module Networks (NMNs) aim at Visual Question Answering (VQA) via composition of modules that tackle a sub-task. NMNs are a promising strategy to achieve systematic generalization, i.e., overcoming biasing factors in the training distribution. However, the aspects of NMNs that facilitate systematic generalization are not fully understood. In this paper, we demonstrate that the degree of modularity of the NMN have large influence on systematic generalization. In a series of experiments on three VQA datasets (VQA-MNIST, SQOOP, and CLEVR-CoGenT), our results reveal that tuning the degree of modularity, especially at the image encoder stage, reaches substantially higher systematic generalization. These findings lead to new NMN architectures that outperform previous ones in terms of systematic generalization.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

22 of 27 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  4. 2015VQA: Visual Question Answeringcited by 2,489
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  6. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  7. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  8. 2015Neural Module Networkscited by 375
  9. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  10. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  11. 2018Compositional Attention Networks for Machine Reasoningcited by 183
  12. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
10 more
  1. 2017Learning to Reason: End-to-End Module Networks for Visual Question Answeringcited by 125
  2. 2019Task-Driven Modular Networks for Zero-Shot Compositional Learningcited by 121
  3. 2020A Benchmark for Systematic Generalization in Grounded Language Understandingcited by 108
  4. 2020On the Value of Out-of-Distribution Testing: An Example of Goodhart's Lawcited by 74
  5. 2019CLOSURE: Assessing Systematic Generalization of CLEVR Modelscited by 63
  6. 2016Visual Question Answering: Datasets, Algorithms, and Future Challengescited by 56
  7. 2018Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networkscited by 51
  8. 2020When and how CNNs generalize to out-of-distribution category-viewpoint combinationscited by 22
  9. 2018Neural Compositional Denotational Semantics for Question Answeringcited by 11
  10. 2020Robustness to Transformations Across Categories: Is Robustness To Transformations Driven by Invariant Neural Representations?cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 4; 2025: 0; 2026: 3.

15 papers.

Sort:

By area:

  1. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  2. 2024A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practicescited by 15
  3. 2022Improving Systematic Generalization Through Modularity and Augmentationcited by 13
  4. 2024Learning to Compose: Improving Object Centric Learning by Injecting Compositionalitycited by 13
  5. 2022Object-Category Aware Reinforcement Learningcited by 6
  6. 2022Transformer Module Networks for Systematic Generalization in Visual Question Answeringcited by 5
  7. 2024Breaking Neural Network Scaling Laws with Modularitycited by 5
  8. 2023Modularity in Deep Learning: A Surveycited by 3
  9. 2026Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignmentcited by 3
  10. 2023D3: Data Diversity Design for Systematic Generalization in Visual Question Answeringcited by 2
  11. 2024DNN Modularization via Activation-Driven Trainingcited by 2
  12. 2023HICO-DET-SG and V-COCO-SG: New Data Splits for Evaluating the Systematic Generalization Performance of Human-Object Interaction Detection Modelscited by 1
3 more
  1. 2026What You Can't See Is What You Learn: Slot-Selective Evidence Masking Favors Compositional Generalization in Shared-Genome Language-Model Societiescited by 1
  2. 2023Modularity Trumps Invariance for Compositional Robustnesscited by 0
  3. 2026Disentanglement-Based Equivariant Learning for Compositional VQAcited by 0