# Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction

arXiv: https://arxiv.org/abs/1511.05756 · HTML: https://mlsanity.com/paper/1511.05756/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2015.

## Built on by (most cited first, 26 of 26)

- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Going Beyond Linear Transformers with Recurrent Fast Weight Programmers](https://mlsanity.com/paper/2106.06295/) (2021)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback](https://mlsanity.com/paper/2106.04128/) (2021)
- [RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Network](https://mlsanity.com/paper/2104.03015/) (2021)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [What data do we need for training an AV motion planner?](https://mlsanity.com/paper/2105.12337/) (2021)
- [VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges](https://mlsanity.com/paper/2212.13296/) (2022)
- [Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answering](https://mlsanity.com/paper/2306.16713/) (2023)
- [Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions](https://mlsanity.com/paper/2304.03147/) (2023)
- [Fine-Grained Captioning of Long Videos through Scene Graph Consolidation](https://mlsanity.com/paper/2502.16427/) (2025)
- [From Pixels to Objects: Cubic Visual Attention for Visual Question Answering](https://mlsanity.com/paper/2206.01923/) (2022)
- [Language Guided Local Infiltration for Interactive Image Retrieval](https://mlsanity.com/paper/2304.07747/) (2023)
- [NeRD: Neural Representation of Distribution for Medical Image Segmentation](https://mlsanity.com/paper/2103.04020/) (2021)
- [Training and challenging models for text-guided fashion image retrieval](https://mlsanity.com/paper/2204.11004/) (2022)
- [Multi-Modal Representation Learning with Text-Driven Soft Masks](https://mlsanity.com/paper/2304.00719/) (2023)
- [Learning Sparse Mixture of Experts for Visual Question Answering](https://mlsanity.com/paper/1909.09192/) (2019)
- [Cross-modal Image Retrieval with Deep Mutual Information Maximization](https://mlsanity.com/paper/2103.06032/) (2021)
- [Adaptive loose optimization for robust question answering](https://mlsanity.com/paper/2305.03971/) (2023)
- [Towards Better Multi-modal Keyphrase Generation via Visual Entity Enhancement and Multi-granularity Image Noise Filtering](https://mlsanity.com/paper/2309.04734/) (2023)
- [Towards Flexible Evaluation for Generative Visual Question Answering](https://mlsanity.com/paper/2408.00300/) (2024)
- [Quantity over Quality: Training an AV Motion Planner with Large Scale Commodity Vision Data](https://mlsanity.com/paper/2203.01681/) (2022)
- [Multimodal Query-guided Object Localization](https://mlsanity.com/paper/2212.00749/) (2022)
- [Bridge Damage Cause Estimation Using Multiple Images Based on Visual Question Answering](https://mlsanity.com/paper/2302.09208/) (2023)
- [On the Cognition of Visual Question Answering Models and Human Intelligence: A Comparative Study](https://mlsanity.com/paper/2310.02528/) (2023)

## Abstract (arXiv)

We tackle image question answering (ImageQA) problem by learning a convolutional neural network (CNN) with a dynamic parameter layer whose weights are determined adaptively based on questions. For the adaptive parameter prediction, we employ a separate parameter prediction network, which consists of gated recurrent unit (GRU) taking a question as its input and a fully-connected layer generating a set of candidate weights as its output. However, it is challenging to construct a parameter prediction network for a large number of parameters in the fully-connected dynamic parameter layer of the CNN. We reduce the complexity of this problem by incorporating a hashing technique, where the candidate weights given by the parameter prediction network are selected using a predefined hash function to determine individual weights in the dynamic parameter layer. The proposed network---joint network with the CNN for ImageQA and the parameter prediction network---is trained end-to-end through back-propagation, where its weights are initialized using a pre-trained CNN and GRU. The proposed algorithm illustrates the state-of-the-art performance on all available public ImageQA benchmarks.