# Explainable Neural Computation via Stack Neural Module Networks

arXiv: https://arxiv.org/abs/1807.08556 · HTML: https://mlsanity.com/paper/1807.08556/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2018.

## Built on by (most cited first, 30 of 54)

- [Visual Programming: Compositional visual reasoning without training](https://mlsanity.com/paper/2211.11559/) (2022)
- [Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models](https://mlsanity.com/paper/2304.09842/) (2023)
- [Neurons in Large Language Models: Dead, N-gram, Positional](https://mlsanity.com/paper/2309.04827/) (2023)
- [Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning](https://mlsanity.com/paper/2103.16564/) (2021)
- [Video as Conditional Graph Hierarchy for Multi-Granular Question Answering](https://mlsanity.com/paper/2112.06197/) (2021)
- [Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language](https://mlsanity.com/paper/2110.15358/) (2021)
- [Zero-Shot Video Question Answering with Procedural Programs](https://mlsanity.com/paper/2312.00937/) (2023)
- [Greedy Gradient Ensemble for Robust Visual Question Answering](https://mlsanity.com/paper/2107.12651/) (2021)
- [Vision-Language Intelligence: Tasks, Representation Learning, and Large Models](https://mlsanity.com/paper/2203.01922/) (2022)
- [TANGO: Training-free Embodied AI Agents for Open-world Tasks](https://mlsanity.com/paper/2412.10402/) (2024)
- [Learning by Planning: Language-Guided Global Image Editing](https://mlsanity.com/paper/2106.13156/) (2021)
- [Separating Skills and Concepts for Novel Visual Question Answering](https://mlsanity.com/paper/2107.09106/) (2021)
- [Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsules](https://mlsanity.com/paper/2105.04836/) (2021)
- [ProTo: Program-Guided Transformer for Program-Guided Tasks](https://mlsanity.com/paper/2110.00804/) (2021)
- [FALCON: Fast Visual Concept Learning by Integrating Images, Linguistic descriptions, and Conceptual Relations](https://mlsanity.com/paper/2203.16639/) (2022)
- [Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA](https://mlsanity.com/paper/2403.16385/) (2024)
- [LPF: A Language-Prior Feedback Objective Function for De-biased Visual Question Answering](https://mlsanity.com/paper/2105.14300/) (2021)
- [Towards Language-guided Visual Recognition via Dynamic Convolutions](https://mlsanity.com/paper/2110.08797/) (2021)
- [3D-Aware Visual Question Answering about Parts, Poses and Occlusions](https://mlsanity.com/paper/2310.17914/) (2023)
- [Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images](https://mlsanity.com/paper/2110.00519/) (2021)
- [Summarization Programs: Interpretable Abstractive Summarization with Neural Modular Trees](https://mlsanity.com/paper/2209.10492/) (2022)
- [VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges](https://mlsanity.com/paper/2212.13296/) (2022)
- [On The Specialization of Neural Modules](https://mlsanity.com/paper/2409.14981/) (2024)
- [REX: Reasoning-aware and Grounded Explanation](https://mlsanity.com/paper/2203.06107/) (2022)
- [VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks](https://mlsanity.com/paper/2104.07921/) (2021)
- [SimVQA: Exploring Simulated Environments for Visual Question Answering](https://mlsanity.com/paper/2203.17219/) (2022)
- [A Comprehensive Survey on Visual Question Answering Datasets and Algorithms](https://mlsanity.com/paper/2411.11150/) (2024)
- [Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering](https://mlsanity.com/paper/2209.06954/) (2022)
- [Prospective Role of Foundation Models in Advancing Autonomous Vehicles](https://mlsanity.com/paper/2405.02288/) (2023)
- [Multimodal Dialogue State Tracking](https://mlsanity.com/paper/2206.07898/) (2022)

## Abstract (arXiv)

In complex inferential tasks like question answering, machine learning models must confront two challenges: the need to implement a compositional reasoning process, and, in many applications, the need for this reasoning process to be interpretable to assist users in both development and prediction. Existing models designed to produce interpretable traces of their decision-making process typically require these traces to be supervised at training time. In this paper, we present a novel neural modular approach that performs compositional reasoning by automatically inducing a desired sub-task decomposition without relying on strong supervision. Our model allows linking different reasoning tasks though shared modules that handle common routines across tasks. Experiments show that the model is more interpretable to human evaluators compared to other state-of-the-art models: users can better understand the model's underlying reasoning procedure and predict when it will succeed or fail based on observing its intermediate outputs.