mlsanityData to 7 Oct 2026

TVQA: Localized, Compositional Video Question Answering

arXiv 1809.01696 ↗First posted 5 Sep 2018cs.CL cs.AI cs.CVcited by 463

Not on the map. Why?

Cited by 65 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent years have witnessed an increasing interest in image-based question-answering (QA) tasks. However, due to data limitations, there has been much less work on video-based QA. In this paper, we present TVQA, a large-scale video QA dataset based on 6 popular TV shows. TVQA consists of 152,545 QA pairs from 21,793 clips, spanning over 460 hours of video. Questions are designed to be compositional in nature, requiring systems to jointly localize relevant moments within a clip, comprehend subtitle-based dialogue, and recognize relevant visual concepts. We provide analyses of this new dataset as well as several baselines and a multi-stream end-to-end trainable neural network framework for the TVQA task. The dataset is publicly available at http://tvqa.cs.unc.edu.

Key facts. Cited by 463 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 74; 2022: 70; 2023: 82; 2024: 89; 2025: 99; 2026: 49.

463 papers.

Sort:

By area:

  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  2. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  3. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  4. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  5. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  6. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  7. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  8. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  9. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  10. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  11. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  12. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
38 more
  1. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  2. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  3. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  4. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  5. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  6. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  7. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
  8. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  9. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  10. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  11. 2023Video Understanding with Large Language Models: A Surveycited by 203
  12. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
  13. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  14. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  15. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  16. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
  17. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  18. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  19. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  20. 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 161
  21. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  22. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  23. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  24. 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
  25. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
  26. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  27. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  28. 2022Revealing Single Frame Bias for Video-and-Language Learningcited by 131
  29. 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 131
  30. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  31. 2022Language Models with Image Descriptors are Strong Few-Shot Video-Language Learnerscited by 117
  32. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  33. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  34. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 112
  35. 2024HourVideo: 1-Hour Video-Language Understandingcited by 110
  36. 2021WebQA: Multihop and Multimodal QAcited by 109
  37. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  38. 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102

50 most cited of 463 All 463 (JSON)