# Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering

arXiv: https://arxiv.org/abs/1803.08896 · HTML: https://mlsanity.com/paper/1803.08896/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2018.

## Built on by (most cited first, 12 of 12)

- [Learning by Self-Explaining](https://mlsanity.com/paper/2309.08395/) (2023)
- [VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges](https://mlsanity.com/paper/2212.13296/) (2022)
- [Hopper: Multi-hop Transformer for Spatiotemporal Reasoning](https://mlsanity.com/paper/2103.10574/) (2021)
- [Domain-robust VQA with diverse datasets and methods but no target labels](https://mlsanity.com/paper/2103.15974/) (2021)
- [Can Pretrained Language Models (Yet) Reason Deductively?](https://mlsanity.com/paper/2210.06442/) (2022)
- [NEWSKVQA: Knowledge-Aware News Video Question Answering](https://mlsanity.com/paper/2202.04015/) (2022)
- [HySTER: A Hybrid Spatio-Temporal Event Reasoner](https://mlsanity.com/paper/2101.06644/) (2021)
- [A Survey on Interpretable Cross-modal Reasoning](https://mlsanity.com/paper/2309.01955/) (2023)
- [Stitching Gaps: Fusing Situated Perceptual Knowledge with Vision Transformers for High-Level Image Classification](https://mlsanity.com/paper/2402.19339/) (2024)
- [MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning](https://mlsanity.com/paper/2605.25842/) (2026)
- [SubGraph Networks based Entity Alignment for Cross-lingual Knowledge Graph](https://mlsanity.com/paper/2205.03557/) (2022)
- [NLKI: A lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks](https://mlsanity.com/paper/2508.19724/) (2025)

## Abstract (arXiv)

Many vision and language tasks require commonsense reasoning beyond data-driven image and natural language processing. Here we adopt Visual Question Answering (VQA) as an example task, where a system is expected to answer a question in natural language about an image. Current state-of-the-art systems attempted to solve the task using deep neural architectures and achieved promising performance. However, the resulting systems are generally opaque and they struggle in understanding questions for which extra knowledge is required. In this paper, we present an explicit reasoning layer on top of a set of penultimate neural network based systems. The reasoning layer enables reasoning and answering questions where additional knowledge is required, and at the same time provides an interpretable interface to the end users. Specifically, the reasoning layer adopts a Probabilistic Soft Logic (PSL) based engine to reason over a basket of inputs: visual relations, the semantic parse of the question, and background ontological knowledge from word2vec and ConceptNet. Experimental analysis of the answers and the key evidential predicates generated on the VQA dataset validate our approach.