# Question-Guided Hybrid Convolution for Visual Question Answering

arXiv: https://arxiv.org/abs/1808.02632 · HTML: https://mlsanity.com/paper/1808.02632/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2018.

## Built on by (most cited first, 12 of 12)

- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning](https://mlsanity.com/paper/2110.13214/) (2021)
- [Fast Convergence of DETR with Spatially Modulated Co-Attention](https://mlsanity.com/paper/2101.07448/) (2021)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Towards Language-guided Visual Recognition via Dynamic Convolutions](https://mlsanity.com/paper/2110.08797/) (2021)
- [Recent, rapid advancement in visual question answering architecture: a review](https://mlsanity.com/paper/2203.01322/) (2022)
- [Bilateral Cross-Modality Graph Matching Attention for Feature Fusion in Visual Question Answering](https://mlsanity.com/paper/2112.07270/) (2021)
- [Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trends](https://mlsanity.com/paper/2401.07518/) (2024)
- [DM2S2: Deep Multi-Modal Sequence Sets with Hierarchical Modality Attention](https://mlsanity.com/paper/2209.03126/) (2022)
- [Fast Convergence of DETR with Spatially Modulated Co-Attention](https://mlsanity.com/paper/2108.02404/) (2021)

## Abstract (arXiv)

In this paper, we propose a novel Question-Guided Hybrid Convolution (QGHC) network for Visual Question Answering (VQA). Most state-of-the-art VQA methods fuse the high-level textual and visual features from the neural network and abandon the visual spatial information when learning multi-modal features.To address these problems, question-guided kernels generated from the input question are designed to convolute with visual features for capturing the textual and visual relationship in the early stage. The question-guided convolution can tightly couple the textual and visual information but also introduce more parameters when learning kernels. We apply the group convolution, which consists of question-independent kernels and question-dependent kernels, to reduce the parameter size and alleviate over-fitting. The hybrid convolution can generate discriminative multi-modal features with fewer parameters. The proposed approach is also complementary to existing bilinear pooling fusion and attention based VQA methods. By integrating with them, our method could further boost the performance. Extensive experiments on public VQA datasets validate the effectiveness of QGHC.