mlsanityData to 7 Oct 2026

ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering

arXiv 1906.02467 ↗First posted 6 Jun 2019cs.CVcited by 559

Not on the map. Why?

Cited by 210 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent developments in modeling language and vision have been successfully applied to image question answering. It is both crucial and natural to extend this research direction to the video domain for video question answering (VideoQA). Compared to the image domain where large scale and fully annotated benchmark datasets exists, VideoQA datasets are limited to small scale and are automatically generated, etc. These limitations restrict their applicability in practice. Here we introduce ActivityNet-QA, a fully annotated and large scale VideoQA dataset. The dataset consists of 58,000 QA pairs on 5,800 complex web videos derived from the popular ActivityNet dataset. We present a statistical analysis of our ActivityNet-QA dataset and conduct extensive experiments on it by comparing existing VideoQA baselines. Moreover, we explore various video representation strategies to improve VideoQA performance, especially for long videos. The dataset is available at https://github.com/MILVLG/activitynet-qa

Key facts. Cited by 559 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 32; 2023: 53; 2024: 156; 2025: 189; 2026: 108.

559 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  4. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  5. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  6. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  7. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  8. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  9. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  10. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  11. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  12. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
38 more
  1. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  2. 2024Long Context Transfer from Language to Visioncited by 502
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  4. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
  5. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  6. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 378
  7. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  8. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
  9. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  10. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
  11. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  12. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  13. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  14. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298
  15. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  16. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  17. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  18. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
  19. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  20. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  21. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  22. 2023Video Understanding with Large Language Models: A Surveycited by 203
  23. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  24. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  25. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
  26. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Modelscited by 177
  27. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  28. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  29. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  30. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
  31. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMscited by 153
  32. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 146
  33. 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Rewardcited by 145
  34. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 144
  35. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  36. 2023VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Datasetcited by 138
  37. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  38. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135

50 most cited of 559 All 559 (JSON)