mlsanityData to 14 Oct 2026

TVQA+: Spatio-Temporal Grounding for Video Question Answering

arXiv 1904.11574 ↗First posted 25 Apr 2019cs.CV cs.AI cs.CLcited by 154

Not on the map. Why?

Abstract (arXiv)

We present the task of Spatio-Temporal Video Question Answering, which requires intelligent systems to simultaneously retrieve relevant moments and detect referenced visual concepts (people and objects) to answer natural language questions about videos. We first augment the TVQA dataset with 310.8K bounding boxes, linking depicted objects to visual concepts in questions and answers. We name this augmented version as TVQA+. We then propose Spatio-Temporal Answerer with Grounded Evidence (STAGE), a unified framework that grounds evidence in both spatial and temporal domains to answer questions about videos. Comprehensive experiments and analyses demonstrate the effectiveness of our framework and how the rich annotations in our TVQA+ dataset can contribute to the question answering task. Moreover, by performing this joint task, our model is able to produce insightful and interpretable spatio-temporal attention visualizations. Dataset and code are publicly available at: http: //tvqa.cs.unc.edu, https://github.com/jayleicn/TVQAplus

Key facts. Cited by 154 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 28; 2023: 35; 2024: 24; 2025: 26; 2026: 18.

154 papers.

Sort:

By area:

  1. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  2. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  3. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  4. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  5. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  6. 2022Visual Spatial Reasoningcited by 293
  7. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  8. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  9. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 188
  10. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
  11. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  12. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 135
38 more
  1. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  2. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  3. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 115
  4. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
  5. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 85
  6. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  7. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  8. 2022An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modelingcited by 74
  9. 2021QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queriescited by 72
  10. 2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
  11. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 68
  12. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  13. 2023ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding Systemcited by 65
  14. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 63
  15. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 58
  16. 2021Natural Language Video Localization: A Revisit in Span-based Question Answering Frameworkcited by 55
  17. 2022MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answeringcited by 55
  18. 2021Grounding 'Grounding' in NLPcited by 44
  19. 2021VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputscited by 42
  20. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  21. 20213D Question Answeringcited by 41
  22. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
  23. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  24. 2023FunQA: Towards Surprising Video Comprehensioncited by 35
  25. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  26. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  27. 2021Multimodal Emotion-Cause Pair Extraction in Conversationscited by 31
  28. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  29. 2022CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Groundingcited by 27
  30. 2022A Dataset for Medical Instructional Video Classification and Question Answeringcited by 26
  31. 2023LvBench: A Benchmark for Long-form Video Understanding with Versatile Multi-modal Question Answeringcited by 25
  32. 2024Understanding Long Videos with Multimodal Language Modelscited by 25
  33. 2023Connecting Vision and Language with Video Localized Narrativescited by 24
  34. 2024Assessing Modality Bias in Video Question Answering Benchmarks with Multimodal Large Language Modelscited by 23
  35. 2021Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsulescited by 22
  36. 2023Tem-adapter: Adapting Image-Text Pretraining for Video Question Answercited by 22
  37. 2021Video-Text Pre-training with Learned Regionscited by 20
  38. 2021Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inferencecited by 18

50 most cited of 154 All 154 (JSON)