mlsanityData to 7 Oct 2026

NEWSKVQA: Knowledge-Aware News Video Question Answering

arXiv 2202.04015 ↗First posted 8 Feb 2022cs.CV cs.MMcited by 4

Not on the map. Why?

Abstract (arXiv)

Answering questions in the context of videos can be helpful in video indexing, video retrieval systems, video summarization, learning management systems and surveillance video analysis. Although there exists a large body of work on visual question answering, work on video question answering (1) is limited to domains like movies, TV shows, gameplay, or human activity, and (2) is mostly based on common sense reasoning. In this paper, we explore a new frontier in video question answering: answering knowledge-based questions in the context of news videos. To this end, we curate a new dataset of 12K news videos spanning across 156 hours with 1M multiple-choice question-answer pairs covering 8263 unique entities. We make the dataset publicly available. Using this dataset, we propose a novel approach, NEWSKVQA (Knowledge-Aware News Video Question Answering) which performs multi-modal inferencing over textual multiple-choice questions, videos, their transcripts and knowledge base, and presents a strong baseline.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 8 Feb 2022.

Builds on

28 of 35 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2015VQA: Visual Question Answeringcited by 2,489
  5. 2019Towards VQA Models That Can Readcited by 1,696
  6. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  7. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  8. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  9. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  10. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  11. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  12. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
16 more
  1. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  2. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  3. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153
  4. 2018TallyQA: Answering Complex Counting Questionscited by 152
  5. 2019Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answeringcited by 146
  6. 2017Learning to Reason: End-to-End Module Networks for Visual Question Answeringcited by 125
  7. 2015Explicit Knowledge-based Reasoning for Visual Question Answeringcited by 85
  8. 2017DeepStory: Video Story QA by Deep Embedded Memory Networkscited by 67
  9. 2016Leveraging Video Descriptions to Learn Video Question Answeringcited by 55
  10. 2019KnowIT VQA: Answering Knowledge-Based Questions about Videoscited by 53
  11. 2015Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sourcescited by 42
  12. 2016MarioQA: Answering Questions by Watching Gameplay Videoscited by 42
  13. 2016A dataset and exploration of models for understanding video data through fill-in-the-blank question-answeringcited by 41
  14. 2018R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answeringcited by 23
  15. 2019Progressive Attention Memory Network for Movie Story Question Answeringcited by 21
  16. 2018Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answeringcited by 12

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2.

4 papers.

  1. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  2. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  3. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  4. 2022Watching the News: Towards VideoQA Models that can Readcited by 20