mlsanityData to 14 Oct 2026

Neural Module Networks

arXiv 1511.02799 ↗First posted 9 Nov 2015cs.CV cs.CL cs.LGcited by 376

Not on the map. Why?

Cited by 48 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual question answering is fundamentally compositional in nature---a question like "where is the dog?" shares substructure with questions like "what color is the dog?" and "where is the cat?" This paper seeks to simultaneously exploit the representational capacity of deep networks and the compositional linguistic structure of questions. We describe a procedure for constructing and learning *neural module networks*, which compose collections of jointly-trained neural "modules" into deep networks for question answering. Our approach decomposes questions into their linguistic substructures, and uses these structures to dynamically instantiate modular networks (with reusable components for recognizing dogs, classifying colors, etc.). The resulting compound networks are jointly trained. We evaluate our approach on two challenging datasets for visual question answering, achieving state-of-the-art results on both the VQA natural image dataset and a new dataset of complex questions about abstract shapes.

Key facts. Cited by 376 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 2; 2020: 0; 2021: 82; 2022: 80; 2023: 79; 2024: 44; 2025: 50; 2026: 38.

376 papers.

Sort:

By area:

  1. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  2. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  3. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  4. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  5. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 334
  6. 2022Visual Spatial Reasoningcited by 293
  7. 2022Binding Language Models in Symbolic Languagescited by 228
  8. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  9. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 181
  10. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
  11. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 164
  12. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
38 more
  1. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 140
  2. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 133
  3. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  4. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  5. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  6. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  7. 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
  8. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 102
  9. 2021Skill Induction and Planning with Latent Languagecited by 99
  10. 2023AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learncited by 97
  11. 2021Medical Visual Question Answering: A Surveycited by 93
  12. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  13. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  14. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
  15. 2022ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Promptscited by 85
  16. 2023Modular Deep Learningcited by 83
  17. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  18. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  19. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  20. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  21. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 75
  22. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 74
  23. 2024MoReVQA: Exploring Modular Reasoning Models for Video Question Answeringcited by 74
  24. 2021Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answeringcited by 71
  25. 2021Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoningcited by 68
  26. 2023AVSegFormer: Audio-Visual Segmentation with Transformercited by 68
  27. 2021Continual Learning via Local Module Compositioncited by 63
  28. 2023Scallop: A Language for Neurosymbolic Programmingcited by 63
  29. 2023NS3D: Neuro-Symbolic Grounding of 3D Objects and Relationscited by 57
  30. 2021Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Languagecited by 55
  31. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  32. 2024Toward Visual Grounding: A Surveycited by 52
  33. 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoningcited by 51
  34. 2023LILO: Learning Interpretable Libraries by Compressing and Documenting Codecited by 49
  35. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 48
  36. 2022Reasoning Like Program Executorscited by 45
  37. 2022Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extractioncited by 44
  38. 2022Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systemscited by 44

50 most cited of 376 All 376 (JSON)