mlsanityData to 7 Oct 2026

Dense-Captioning Events in Videos

arXiv 1705.00754 ↗First posted 2 May 2017cs.CVcited by 789

Not on the map. Why?

Cited by 103 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Most natural videos contain numerous events. For example, in a video of a "man playing a piano", the video might also contain "another man dancing" or "a crowd clapping". We introduce the task of dense-captioning events, which involves both detecting and describing events in a video. We propose a new model that is able to identify all events in a single pass of the video while simultaneously describing the detected events with natural language. Our model introduces a variant of an existing proposal module that is designed to capture both short as well as long events that span minutes. To capture the dependencies between the events in a video, our model introduces a new captioning module that uses contextual information from past and future events to jointly describe all events. We also introduce ActivityNet Captions, a large-scale benchmark for dense-captioning events. ActivityNet Captions contains 20k videos amounting to 849 video hours with 100k total descriptions, each with it's unique start and end time. Finally, we report performances of our model for dense-captioning events, video retrieval and localization.

Key facts. Cited by 789 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 101; 2022: 118; 2023: 146; 2024: 137; 2025: 156; 2026: 130.

789 papers.

Sort:

By area:

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  3. 2021Transformers in Vision: A Surveycited by 1,009
  4. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  5. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  6. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  7. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  8. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  9. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  10. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  11. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  12. 2023Learning Interactive Real-World Simulatorscited by 440
38 more
  1. 2022Multimodal Learning with Transformers: A Surveycited by 399
  2. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  3. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  4. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  5. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  6. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  7. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  8. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  9. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
  10. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  11. 2022Egocentric Video-Language Pretrainingcited by 231
  12. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  13. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  14. 2023A Survey on Video Diffusion Modelscited by 203
  15. 2023Video Understanding with Large Language Models: A Surveycited by 203
  16. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  17. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  18. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  19. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  20. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  21. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  22. 2023VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Datasetcited by 138
  23. 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 138
  24. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  25. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
  26. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
  27. 2022Revealing Single Frame Bias for Video-and-Language Learningcited by 131
  28. 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 131
  29. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  30. 2023X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languagescited by 127
  31. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  32. 2023PaLI-3 Vision Language Models: Smaller, Faster, Strongercited by 125
  33. 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
  34. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  35. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  36. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 115
  37. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 110
  38. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107

50 most cited of 789 All 789 (JSON)