# Visual Question Answering with Memory-Augmented Networks

arXiv: https://arxiv.org/abs/1707.04968 · HTML: https://mlsanity.com/paper/1707.04968/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2017.

## Built on by (most cited first, 25 of 25)

- [Remember Intentions: Retrospective-Memory-based Trajectory Prediction](https://mlsanity.com/paper/2203.11474/) (2022)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering](https://mlsanity.com/paper/2104.14085/) (2021)
- [HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detection](https://mlsanity.com/paper/2104.00902/) (2021)
- [Memory-Guided Semantic Learning Network for Temporal Sentence Grounding](https://mlsanity.com/paper/2201.00454/) (2022)
- [SMEMO: Social Memory for Trajectory Forecasting](https://mlsanity.com/paper/2203.12446/) (2022)
- [Cross Attentional Audio-Visual Fusion for Dimensional Emotion Recognition](https://mlsanity.com/paper/2111.05222/) (2021)
- [Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem](https://mlsanity.com/paper/2207.11850/) (2022)
- [Audio-Visual Fusion for Emotion Recognition in the Valence-Arousal Space Using Joint Cross-Attention](https://mlsanity.com/paper/2209.09068/) (2022)
- [MetAug: Contrastive Learning via Meta Feature Augmentation](https://mlsanity.com/paper/2203.05119/) (2022)
- [VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges](https://mlsanity.com/paper/2212.13296/) (2022)
- [S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learning](https://mlsanity.com/paper/2309.02155/) (2023)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [Follow the Timeline! Generating Abstractive and Extractive Timeline Summary in Chronological Order](https://mlsanity.com/paper/2301.00867/) (2023)
- [Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQA](https://mlsanity.com/paper/2312.13594/) (2023)
- [Memory and attention in deep learning](https://mlsanity.com/paper/2107.01390/) (2021)
- [Audio-Visual Speaker Verification via Joint Cross-Attention](https://mlsanity.com/paper/2309.16569/) (2023)
- [Relation Regularized Scene Graph Generation](https://mlsanity.com/paper/2202.10826/) (2022)
- [From Shallow to Deep: Compositional Reasoning over Graphs for Visual Question Answering](https://mlsanity.com/paper/2206.12533/) (2022)
- [Multi-Clue Reasoning with Memory Augmentation for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2312.12723/) (2023)
- [Memory Wrap: a Data-Efficient and Interpretable Extension to Image Classification Models](https://mlsanity.com/paper/2106.01440/) (2021)
- [Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System](https://mlsanity.com/paper/2207.07827/) (2022)
- [Text-Aware Dual Routing Network for Visual Question Answering](https://mlsanity.com/paper/2211.14450/) (2022)
- [Cross-Modal Reasoning with Event Correlation for Video Question Answering](https://mlsanity.com/paper/2312.12721/) (2023)
- [Retrieval-Augmented Natural Language Reasoning for Explainable Visual Question Answering](https://mlsanity.com/paper/2408.17006/) (2024)

## Abstract (arXiv)

In this paper, we exploit a memory-augmented neural network to predict accurate answers to visual questions, even when those answers occur rarely in the training set. The memory network incorporates both internal and external memory blocks and selectively pays attention to each training exemplar. We show that memory-augmented neural networks are able to maintain a relatively long-term memory of scarce training exemplars, which is important for visual question answering due to the heavy-tailed distribution of answers in a general VQA setting. Experimental results on two large-scale benchmark datasets show the favorable performance of the proposed algorithm with a comparison to state of the art.