# NEWSKVQA: Knowledge-Aware News Video Question Answering

arXiv: https://arxiv.org/abs/2202.04015 · HTML: https://mlsanity.com/paper/2202.04015/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [From Recognition to Cognition: Visual Commonsense Reasoning](https://mlsanity.com/paper/1811.10830/) (2018)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering](https://mlsanity.com/paper/1704.04497/) (2017)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [FVQA: Fact-based Visual Question Answering](https://mlsanity.com/paper/1606.05433/) (2016)
- [TVQA+: Spatio-Temporal Grounding for Video Question Answering](https://mlsanity.com/paper/1904.11574/) (2019)
- [TallyQA: Answering Complex Counting Questions](https://mlsanity.com/paper/1810.12440/) (2018)
- [Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering](https://mlsanity.com/paper/1904.04357/) (2019)
- [Learning to Reason: End-to-End Module Networks for Visual Question Answering](https://mlsanity.com/paper/1704.05526/) (2017)
- [Explicit Knowledge-based Reasoning for Visual Question Answering](https://mlsanity.com/paper/1511.02570/) (2015)
- [DeepStory: Video Story QA by Deep Embedded Memory Networks](https://mlsanity.com/paper/1707.00836/) (2017)
- [Leveraging Video Descriptions to Learn Video Question Answering](https://mlsanity.com/paper/1611.04021/) (2016)
- [KnowIT VQA: Answering Knowledge-Based Questions about Videos](https://mlsanity.com/paper/1910.10706/) (2019)
- [Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources](https://mlsanity.com/paper/1511.06973/) (2015)
- [MarioQA: Answering Questions by Watching Gameplay Videos](https://mlsanity.com/paper/1612.01669/) (2016)
- [A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering](https://mlsanity.com/paper/1611.07810/) (2016)
- [R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering](https://mlsanity.com/paper/1805.09701/) (2018)
- [Progressive Attention Memory Network for Movie Story Question Answering](https://mlsanity.com/paper/1904.08607/) (2019)
- [Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering](https://mlsanity.com/paper/1803.08896/) (2018)

## Built on by (most cited first, 4 of 4)

- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://mlsanity.com/paper/2309.15402/) (2023)
- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [Watching the News: Towards VideoQA Models that can Read](https://mlsanity.com/paper/2211.05588/) (2022)

## Abstract (arXiv)

Answering questions in the context of videos can be helpful in video indexing, video retrieval systems, video summarization, learning management systems and surveillance video analysis. Although there exists a large body of work on visual question answering, work on video question answering (1) is limited to domains like movies, TV shows, gameplay, or human activity, and (2) is mostly based on common sense reasoning. In this paper, we explore a new frontier in video question answering: answering knowledge-based questions in the context of news videos. To this end, we curate a new dataset of 12K news videos spanning across 156 hours with 1M multiple-choice question-answer pairs covering 8263 unique entities. We make the dataset publicly available. Using this dataset, we propose a novel approach, NEWSKVQA (Knowledge-Aware News Video Question Answering) which performs multi-modal inferencing over textual multiple-choice questions, videos, their transcripts and knowledge base, and presents a strong baseline.