# ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering

arXiv: https://arxiv.org/abs/1511.05960 · HTML: https://mlsanity.com/paper/1511.05960/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2015.

## Built on by (most cited first, 24 of 24)

- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data](https://mlsanity.com/paper/2205.03147/) (2022)
- [SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering](https://mlsanity.com/paper/2204.02285/) (2022)
- [Attention Mechanism in Neural Networks: Where it Comes and Where it Goes](https://mlsanity.com/paper/2204.13154/) (2022)
- [VQA-Aid: Visual Question Answering for Post-Disaster Damage Assessment and Analysis](https://mlsanity.com/paper/2106.10548/) (2021)
- [Understanding the computational demands underlying visual reasoning](https://mlsanity.com/paper/2108.03603/) (2021)
- [ViOCRVQA: Novel Benchmark Dataset and Vision Reader for Visual Question Answering by Understanding Vietnamese Text in Images](https://mlsanity.com/paper/2404.18397/) (2024)
- [A Comprehensive Survey on Visual Question Answering Datasets and Algorithms](https://mlsanity.com/paper/2411.11150/) (2024)
- [How to find a good image-text embedding for remote sensing visual question answering?](https://mlsanity.com/paper/2109.11848/) (2021)
- [EAML: Ensemble Self-Attention-based Mutual Learning Network for Document Image Classification](https://mlsanity.com/paper/2305.06923/) (2023)
- [Leveraging Constraint Programming in a Deep Learning Approach for Dynamically Solving the Flexible Job-Shop Scheduling Problem](https://mlsanity.com/paper/2403.09249/) (2024)
- [TexRel: a Green Family of Datasets for Emergent Communications on Relations](https://mlsanity.com/paper/2105.12804/) (2021)
- [Learning Sparse Mixture of Experts for Visual Question Answering](https://mlsanity.com/paper/1909.09192/) (2019)
- [Multi-Clue Reasoning with Memory Augmentation for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2312.12723/) (2023)
- [Improved RAMEN: Towards Domain Generalization for Visual Question Answering](https://mlsanity.com/paper/2109.02370/) (2021)
- [Towards Unsupervised Visual Reasoning: Do Off-The-Shelf Features Know How to Reason?](https://mlsanity.com/paper/2212.10292/) (2022)
- [Revising deep learning methods in parking lot occupancy detection](https://mlsanity.com/paper/2306.04288/) (2023)
- [PhD Thesis: Exploring the role of (self-)attention in cognitive and computer vision architecture](https://mlsanity.com/paper/2306.14650/) (2023)
- [Attention to Entropic Communication](https://mlsanity.com/paper/2307.11423/) (2023)
- [Cross-Modal Reasoning with Event Correlation for Video Question Answering](https://mlsanity.com/paper/2312.12721/) (2023)
- [Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison](https://mlsanity.com/paper/2502.14827/) (2025)
- [Machine Intelligence that Understands Visual and Linguistic Information and Interacts with Humans and Environments](https://mlsanity.com/paper/2605.24020/) (2026)

## Abstract (arXiv)

We propose a novel attention based deep learning architecture for visual question answering task (VQA). Given an image and an image related natural language question, VQA generates the natural language answer for the question. Generating the correct answers requires the model's attention to focus on the regions corresponding to the question, because different questions inquire about the attributes of different image regions. We introduce an attention based configurable convolutional neural network (ABC-CNN) to learn such question-guided attention. ABC-CNN determines an attention map for an image-question pair by convolving the image feature map with configurable convolutional kernels derived from the question's semantics. We evaluate the ABC-CNN architecture on three benchmark VQA datasets: Toronto COCO-QA, DAQUAR, and VQA dataset. ABC-CNN model achieves significant improvements over state-of-the-art methods on these datasets. The question-guided attention generated by ABC-CNN is also shown to reflect the regions that are highly relevant to the questions.