# From Pixels to Objects: Cubic Visual Attention for Visual Question Answering

arXiv: https://arxiv.org/abs/2206.01923 · HTML: https://mlsanity.com/paper/2206.01923/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Visual7W: Grounded Question Answering in Images](https://mlsanity.com/paper/1511.03416/) (2015)
- [Hierarchical Question-Image Co-Attention for Visual Question Answering](https://mlsanity.com/paper/1606.00061/) (2016)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [Exploring Models and Data for Image Question Answering](https://mlsanity.com/paper/1505.02074/) (2015)
- [Dual Attention Networks for Multimodal Reasoning and Matching](https://mlsanity.com/paper/1611.00471/) (2016)
- [Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering](https://mlsanity.com/paper/1505.05612/) (2015)
- [Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering](https://mlsanity.com/paper/1511.05234/) (2015)
- [Where To Look: Focus Regions for Visual Question Answering](https://mlsanity.com/paper/1511.07394/) (2015)
- [Dynamic Memory Networks for Visual and Textual Question Answering](https://mlsanity.com/paper/1603.01417/) (2016)
- [Ask Your Neurons: A Neural-based Approach to Answering Questions about Images](https://mlsanity.com/paper/1505.01121/) (2015)
- [Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources](https://mlsanity.com/paper/1511.06973/) (2015)
- [Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction](https://mlsanity.com/paper/1511.05756/) (2015)
- [Learning to Answer Questions From Image Using Convolutional Neural Network](https://mlsanity.com/paper/1506.00333/) (2015)
- [Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning](https://mlsanity.com/paper/1706.01231/) (2017)
- [Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering](https://mlsanity.com/paper/1711.06794/) (2017)
- [The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions](https://mlsanity.com/paper/1612.05386/) (2016)
- [A Focused Dynamic Attention Model for Visual Question Answering](https://mlsanity.com/paper/1604.01485/) (2016)

## Built on by (most cited first, 7 of 7)

- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusion](https://mlsanity.com/paper/2408.07303/) (2024)
- [A Comprehensive Survey on Visual Question Answering Datasets and Algorithms](https://mlsanity.com/paper/2411.11150/) (2024)
- [HGAN: Hierarchical Graph Alignment Network for Image-Text Retrieval](https://mlsanity.com/paper/2212.08281/) (2022)
- [LOIS: Looking Out of Instance Semantics for Visual Question Answering](https://mlsanity.com/paper/2307.14142/) (2023)
- [Object Attribute Matters in Visual Question Answering](https://mlsanity.com/paper/2401.09442/) (2023)
- [DiN: Diffusion Model for Robust Medical VQA with Semantic Noisy Labels](https://mlsanity.com/paper/2503.18536/) (2025)

## Abstract (arXiv)

Recently, attention-based Visual Question Answering (VQA) has achieved great success by utilizing question to selectively target different visual areas that are related to the answer. Existing visual attention models are generally planar, i.e., different channels of the last conv-layer feature map of an image share the same weight. This conflicts with the attention mechanism because CNN features are naturally spatial and channel-wise. Also, visual attention models are usually conducted on pixel-level, which may cause region discontinuous problems. In this paper, we propose a Cubic Visual Attention (CVA) model by successfully applying a novel channel and spatial attention on object regions to improve VQA task. Specifically, instead of attending to pixels, we first take advantage of the object proposal networks to generate a set of object candidates and extract their associated conv features. Then, we utilize the question to guide channel attention and spatial attention calculation based on the con-layer feature map. Finally, the attended visual features and the question are combined to infer the answer. We assess the performance of our proposed CVA on three public image QA datasets, including COCO-QA, VQA and Visual7W. Experimental results show that our proposed method significantly outperforms the state-of-the-arts.