mlsanityData to 7 Oct 2026

KnowIT VQA: Answering Knowledge-Based Questions about Videos

arXiv 1910.10706 ↗First posted 23 Oct 2019cs.CV cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

We propose a novel video understanding task by fusing knowledge-based and video question answering. First, we introduce KnowIT VQA, a video dataset with 24,282 human-generated question-answer pairs about a popular sitcom. The dataset combines visual, textual and temporal coherence reasoning together with knowledge-based questions, which need of the experience obtained from the viewing of the series to be answered. Second, we propose a video understanding model by combining the visual and textual video content with specific knowledge about the show. Our main findings are: (i) the incorporation of knowledge produces outstanding improvements for VQA in video, and (ii) the performance on KnowIT VQA still lags well behind human accuracy, indicating its usefulness for studying current video modelling limitations.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 13; 2023: 7; 2024: 7; 2025: 13; 2026: 2.

53 papers.

Sort:

By area:

  1. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  2. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  3. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  4. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
  5. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  6. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  7. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  8. 2022REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answeringcited by 58
  9. 20213D Question Answeringcited by 41
  10. 2024Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sportscited by 41
  11. 2022Gender and Racial Bias in Visual Question Answering Datasetscited by 39
  12. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
38 more
  1. 2023FunQA: Towards Surprising Video Comprehensioncited by 35
  2. 2023Distribution-Aware Prompt Tuning for Vision-Language Modelscited by 35
  3. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  4. 2022i-Code: An Integrative and Composable Multimodal Learning Frameworkcited by 24
  5. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  6. 2022Modern Question Answering Datasets and Benchmarks: A Surveycited by 23
  7. 2023Tem-adapter: Adapting Image-Text Pretraining for Video Question Answercited by 22
  8. 2025ECBench: Can Multi-modal Foundation Models Understand the Egocentric World? A Holistic Embodied Cognition Benchmarkcited by 18
  9. 2025Learning Musical Representations for Music Performance Question Answeringcited by 17
  10. 2022Video Question Answering with Iterative Video-Text Co-Tokenizationcited by 16
  11. 2024WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoningcited by 16
  12. 2024Referring Atomic Video Action Recognitioncited by 16
  13. 2025Knowledge Graph-extended Retrieval Augmented Generation for Question Answeringcited by 14
  14. 2021Recent Advances in Video Question Answering: A Review of Datasets and Methodscited by 12
  15. 2021A survey on VQA_Datasets and Approachescited by 11
  16. 2025Video SimpleQA: Towards Factuality Evaluation in Large Video Language Modelscited by 11
  17. 2022WildQA: In-the-Wild Video Question Answeringcited by 9
  18. 2025TOGA: Temporally Grounded Open-Ended Video QA with Weak Supervisioncited by 8
  19. 2026Demo-ICL: In-Context Learning for Procedural Video Knowledge Acquisitioncited by 8
  20. 2021On the hidden treasure of dialog in video question answeringcited by 7
  21. 2021Transferring Domain-Agnostic Knowledge in Video Question Answeringcited by 7
  22. 2025Can Vision-Language Models Answer Face to Face Questions in the Real-World?cited by 7
  23. 2023Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Promptscited by 5
  24. 2021LiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answeringcited by 4
  25. 2022NEWSKVQA: Knowledge-Aware News Video Question Answeringcited by 4
  26. 2024Answering Diverse Questions via Text Attached with Key Audio-Visual Cluescited by 4
  27. 2021Relation-aware Hierarchical Attention Framework for Video Question Answeringcited by 3
  28. 2024YTCommentQA: Video Question Answerability in Instructional Videoscited by 3
  29. 2023Two is Better Than One: Answering Complex Questions by Multiple Knowledge Sources with Generalized Linkscited by 2
  30. 2025Diversity Covariance-Aware Prompt Learning for Vision-Language Modelscited by 2
  31. 2025Beyond CLIP Generalization: Against Forward&Backward Forgetting Adapter for Continual Learning of Vision-Language Modelscited by 2
  32. 2025CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generationcited by 2
  33. 2021CogME: A Cognition-Inspired Multi-Dimensional Evaluation Metric for Story Understandingcited by 1
  34. 2022Attention in Reasoning: Dataset, Analysis, and Modelingcited by 1
  35. 2021EKTVQA: Generalized use of External Knowledge to empower Scene Text in Text-VQAcited by 0
  36. 2024QTG-VQA: Question-Type-Guided Architectural for VideoQA Systemscited by 0
  37. 2025Outside Knowledge Conversational Video (OKCV) Dataset -- Dialoguing over Videoscited by 0
  38. 2025A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Conceptscited by 0

50 most cited of 53 All 53 (JSON)