# Learning to Answer Questions From Image Using Convolutional Neural Network

arXiv: https://arxiv.org/abs/1506.00333 · HTML: https://mlsanity.com/paper/1506.00333/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2015.

## Built on by (most cited first, 23 of 23)

- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Noisy Correspondence Learning with Meta Similarity Correction](https://mlsanity.com/paper/2304.06275/) (2023)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Robust Visual Question Answering: Datasets, Methods, and Future Challenges](https://mlsanity.com/paper/2307.11471/) (2023)
- [CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval](https://mlsanity.com/paper/2208.09843/) (2022)
- [ExDDV: A New Dataset for Explainable Deepfake Detection in Video](https://mlsanity.com/paper/2503.14421/) (2025)
- [Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions](https://mlsanity.com/paper/2304.03147/) (2023)
- [Boosting Video-Text Retrieval with Explicit High-Level Semantics](https://mlsanity.com/paper/2208.04215/) (2022)
- [ViOCRVQA: Novel Benchmark Dataset and Vision Reader for Visual Question Answering by Understanding Vietnamese Text in Images](https://mlsanity.com/paper/2404.18397/) (2024)
- [Dynamic Key-value Memory Enhanced Multi-step Graph Reasoning for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2203.02985/) (2022)
- [From Pixels to Objects: Cubic Visual Attention for Visual Question Answering](https://mlsanity.com/paper/2206.01923/) (2022)
- [IMoRe: Implicit Program-Guided Reasoning for Human Motion Q&A](https://mlsanity.com/paper/2508.01984/) (2025)
- [Multi-Clue Reasoning with Memory Augmentation for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2312.12723/) (2023)
- [A Curriculum Learning Approach to Reinforcement Learning: Leveraging RAG for Multimodal Question Answering](https://mlsanity.com/paper/2508.10337/) (2025)
- [Improved RAMEN: Towards Domain Generalization for Visual Question Answering](https://mlsanity.com/paper/2109.02370/) (2021)
- [Mathematical Justification of Hard Negative Mining via Isometric Approximation Theorem](https://mlsanity.com/paper/2210.11173/) (2022)
- [SCB-Dataset: A Dataset for Detecting Student and Teacher Classroom Behavior](https://mlsanity.com/paper/2304.02488/) (2023)
- [SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Dataset](https://mlsanity.com/paper/2410.22648/) (2024)
- [Barlow constrained optimization for Visual Question Answering](https://mlsanity.com/paper/2203.03727/) (2022)
- [Bridge Damage Cause Estimation Using Multiple Images Based on Visual Question Answering](https://mlsanity.com/paper/2302.09208/) (2023)
- [GC-KBVQA: A New Four-Stage Framework for Enhancing Knowledge Based Visual Question Answering Performance](https://mlsanity.com/paper/2505.19354/) (2025)
- [GHR-VQA: Graph-guided Hierarchical Relational Reasoning for Video Question Answering](https://mlsanity.com/paper/2511.20201/) (2025)
- [Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models](https://mlsanity.com/paper/2609.04720/) (2026)

## Abstract (arXiv)

In this paper, we propose to employ the convolutional neural network (CNN) for the image question answering (QA). Our proposed CNN provides an end-to-end framework with convolutional architectures for learning not only the image and question representations, but also their inter-modal interactions to produce the answer. More specifically, our model consists of three CNNs: one image CNN to encode the image content, one sentence CNN to compose the words of the question, and one multimodal convolution layer to learn their joint representation for the classification in the space of candidate answer words. We demonstrate the efficacy of our proposed model on the DAQUAR and COCO-QA datasets, which are two benchmark datasets for the image QA, with the performances significantly outperforming the state-of-the-art.