# A Read-Write Memory Network for Movie Story Understanding

arXiv: https://arxiv.org/abs/1709.09345 · HTML: https://mlsanity.com/paper/1709.09345/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2017.

## Built on by (most cited first, 18 of 18)

- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [MV-TON: Memory-based Video Virtual Try-on network](https://mlsanity.com/paper/2108.07502/) (2021)
- [Memory-guided Unsupervised Image-to-image Translation](https://mlsanity.com/paper/2104.05170/) (2021)
- [Hierarchical Self-supervised Representation Learning for Movie Understanding](https://mlsanity.com/paper/2204.03101/) (2022)
- [Multi-Scale Memory-Based Video Deblurring](https://mlsanity.com/paper/2204.02977/) (2022)
- [ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos](https://mlsanity.com/paper/2305.02519/) (2023)
- [Motion-aware Memory Network for Fast Video Salient Object Detection](https://mlsanity.com/paper/2208.00946/) (2022)
- [Beyond Short Clips: End-to-End Video-Level Learning with Collaborative Memories](https://mlsanity.com/paper/2104.01198/) (2021)
- [On the hidden treasure of dialog in video question answering](https://mlsanity.com/paper/2103.14517/) (2021)
- [MILA: Memory-Based Instance-Level Adaptation for Cross-Domain Object Detection](https://mlsanity.com/paper/2309.01086/) (2023)
- [Long Story Short: a Summarize-then-Search Method for Long Video Question Answering](https://mlsanity.com/paper/2311.01233/) (2023)
- [Memory-guided Image De-raining Using Time-Lapse Data](https://mlsanity.com/paper/2201.01883/) (2022)
- [Learning Fine-Grained Visual Understanding for Video Question Answering via Decoupling Spatial-Temporal Modeling](https://mlsanity.com/paper/2210.03941/) (2022)
- [Visual Commonsense based Heterogeneous Graph Contrastive Learning](https://mlsanity.com/paper/2311.06553/) (2023)
- [Visually Similar Pair Alignment for Robust Cross-Domain Object Detection](https://mlsanity.com/paper/2504.06607/) (2025)
- [Adaptive Memory Management for Video Object Segmentation](https://mlsanity.com/paper/2204.06626/) (2022)
- [Frame-Subtitle Self-Supervision for Multi-Modal Video Question Answering](https://mlsanity.com/paper/2209.03609/) (2022)

## Abstract (arXiv)

We propose a novel memory network model named Read-Write Memory Network (RWMN) to perform question and answering tasks for large-scale, multimodal movie story understanding. The key focus of our RWMN model is to design the read network and the write network that consist of multiple convolutional layers, which enable memory read and write operations to have high capacity and flexibility. While existing memory-augmented network models treat each memory slot as an independent block, our use of multi-layered CNNs allows the model to read and write sequential memory cells as chunks, which is more reasonable to represent a sequential story because adjacent memory blocks often have strong correlations. For evaluation, we apply our model to all the six tasks of the MovieQA benchmark, and achieve the best accuracies on several tasks, especially on the visual QA task. Our model shows a potential to better understand not only the content in the story, but also more abstract information, such as relationships between characters and the reasons for their actions.