# Recent Advances in Video Question Answering: A Review of Datasets and Methods

arXiv: https://arxiv.org/abs/2101.05954 · HTML: https://mlsanity.com/paper/2101.05954/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 52 references that are in the map.
- First posted on arXiv 15 Jan 2021.

## Builds on (most cited first, 24 of 24)

- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering](https://mlsanity.com/paper/1704.04497/) (2017)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [A Dataset for Movie Description](https://mlsanity.com/paper/1501.02530/) (2015)
- [Hierarchical Conditional Relation Networks for Video Question Answering](https://mlsanity.com/paper/2002.10698/) (2020)
- [TVQA+: Spatio-Temporal Grounding for Video Question Answering](https://mlsanity.com/paper/1904.11574/) (2019)
- [Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering](https://mlsanity.com/paper/1904.04357/) (2019)
- [TGIF: A New Dataset and Benchmark on Animated GIF Description](https://mlsanity.com/paper/1604.02748/) (2016)
- [Motion-Appearance Co-Memory Networks for Video Question Answering](https://mlsanity.com/paper/1803.10906/) (2018)
- [Location-aware Graph Convolutional Networks for Video Question Answering](https://mlsanity.com/paper/2008.09105/) (2020)
- [DeepStory: Video Story QA by Deep Embedded Memory Networks](https://mlsanity.com/paper/1707.00836/) (2017)
- [End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answering](https://mlsanity.com/paper/1610.02947/) (2016)
- [Leveraging Video Descriptions to Learn Video Question Answering](https://mlsanity.com/paper/1611.04021/) (2016)
- [KnowIT VQA: Answering Knowledge-Based Questions about Videos](https://mlsanity.com/paper/1910.10706/) (2019)
- [DramaQA: Character-Centered Video Story Understanding with Hierarchical QA](https://mlsanity.com/paper/2005.03356/) (2020)
- [MarioQA: Answering Questions by Watching Gameplay Videos](https://mlsanity.com/paper/1612.01669/) (2016)
- [A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering](https://mlsanity.com/paper/1611.07810/) (2016)
- [Video Question Answering via Attribute-Augmented Attention Network Learning](https://mlsanity.com/paper/1707.06355/) (2017)
- [Progressive Attention Memory Network for Movie Story Question Answering](https://mlsanity.com/paper/1904.08607/) (2019)
- [Multimodal Dual Attention Memory for Video Story Question Answering](https://mlsanity.com/paper/1809.07999/) (2018)
- [Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7](https://mlsanity.com/paper/1806.00525/) (2018)
- [Joint Video and Text Parsing for Understanding Events and Answering Queries](https://mlsanity.com/paper/1308.6628/) (2013)
- [Learning Question-Guided Video Representation for Multi-Turn Video Question Answering](https://mlsanity.com/paper/1907.13280/) (2019)

## Built on by (most cited first, 12 of 12)

- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering](https://mlsanity.com/paper/2204.09634/) (2022)
- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Locate before Answering: Answer Guided Question Localization for Video Question Answering](https://mlsanity.com/paper/2210.02081/) (2022)
- [End-to-End Video Question Answering with Frame Scoring Mechanisms and Adaptive Sampling](https://mlsanity.com/paper/2407.15047/) (2024)
- [Learning Situation Hyper-Graphs for Video Question Answering](https://mlsanity.com/paper/2304.08682/) (2023)
- [M2-RAAP: A Multi-Modal Recipe for Advancing Adaptation-based Pre-training towards Effective and Efficient Zero-shot Video-text Retrieval](https://mlsanity.com/paper/2401.17797/) (2024)
- [CogME: A Cognition-Inspired Multi-Dimensional Evaluation Metric for Story Understanding](https://mlsanity.com/paper/2107.09847/) (2021)
- [Ranking Distillation for Open-Ended Video Question Answering with Insufficient Labels](https://mlsanity.com/paper/2403.14430/) (2024)
- [Audiopedia: Audio QA with Knowledge](https://mlsanity.com/paper/2412.20619/) (2024)
- [Beyond Isolated Facts: Synthesizing Narrative and Grounded Supervision for VideoQA](https://mlsanity.com/paper/2509.24445/) (2025)

## Abstract (arXiv)

Video Question Answering (VQA) is a recent emerging challenging task in the field of Computer Vision. Several visual information retrieval techniques like Video Captioning/Description and Video-guided Machine Translation have preceded the task of VQA. VQA helps to retrieve temporal and spatial information from the video scenes and interpret it. In this survey, we review a number of methods and datasets for the task of VQA. To the best of our knowledge, no previous survey has been conducted for the VQA task.