mlsanityData to 14 Oct 2026

Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocols

arXiv 2311.02538 ↗First posted 5 Nov 2023cs.CV cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Untrimmed videos have interrelated events, dependencies, context, overlapping events, object-object interactions, domain specificity, and other semantics that are worth highlighting while describing a video in natural language. Owing to such a vast diversity, a single sentence can only correctly describe a portion of the video. Dense Video Captioning (DVC) aims at detecting and describing different events in a given video. The term DVC originated in the 2017 ActivityNet challenge, after which considerable effort has been made to address the challenge. Dense Video Captioning is divided into three sub-tasks: (1) Video Feature Extraction (VFE), (2) Temporal Event Localization (TEL), and (3) Dense Caption Generation (DCG). This review aims to discuss all the studies that claim to perform DVC along with its sub-tasks and summarize their results. We also discuss all the datasets that have been used for DVC. Lastly, we highlight some emerging challenges and future trends in the field.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 70 of its 154 references that are in the map. First posted on arXiv 5 Nov 2023.

Builds on

70 of 154 references.

References · 70

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  8. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  9. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  10. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  11. 2021ViViT: A Video Vision Transformercited by 1,620
  12. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
38 more
  1. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  2. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  3. 2015Pointer Networkscited by 895
  4. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  5. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  6. 2017Dense-Captioning Events in Videoscited by 793
  7. 2017Localizing Moments in Video with Natural Languagecited by 663
  8. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  9. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  10. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  11. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  12. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 414
  13. 2017Temporal Action Detection with Structured Segment Networkscited by 267
  14. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  15. 2018End-to-End Dense Video Captioning with Masked Transformercited by 214
  16. 2015Semi-supervised Sequence Learningcited by 198
  17. 2020Temporal Pyramid Network for Action Recognitioncited by 163
  18. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
  19. 2017Single Shot Temporal Action Detectioncited by 123
  20. 2020Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentencescited by 118
  21. 2020Multi-modal Dense Video Captioningcited by 113
  22. 2019Fast Learning of Temporal Action Proposal via Dense Boundary Generatorcited by 89
  23. 2019Proposal-free Temporal Moment Localization of a Natural-Language Query in Video using Guided Attentioncited by 81
  24. 2020Multimodal Pretraining for Dense Video Captioningcited by 79
  25. 2020A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformercited by 76
  26. 2022An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modelingcited by 74
  27. 2018Bidirectional Attentive Fusion with Context Gating for Dense Video Captioningcited by 68
  28. 2018Jointly Localizing and Describing Events for Dense Video Captioningcited by 60
  29. 2018Weakly Supervised Dense Event Captioning in Videoscited by 58
  30. 2019Streamlined Dense Video Captioningcited by 57
  31. 2019Watch, Listen and Tell: Multi-modal Weakly Supervised Dense Event Captioningcited by 38
  32. 2018Move Forward and Tell: A Progressive Generator of Video Descriptionscited by 35
  33. 2021Zero-shot Natural Language Video Localizationcited by 34
  34. 2021Deep Learning-based Action Detection in Untrimmed Videos: A Surveycited by 32
  35. 2021A Survey on Temporal Sentence Grounding in Videoscited by 29
  36. 2022Proposal-Free Temporal Action Detection via Global Segmentation Mask Learningcited by 29
  37. 2023A Review of Deep Learning for Video Captioningcited by 28
  38. 2022End-to-end Dense Video Captioning as Sequence Generationcited by 27

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 6.

17 papers.

Sort:

By area:

  1. 2025StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compressioncited by 26
  2. 2025Time Blindness: Why Video-Language Models Can't See What Humans Can?cited by 18
  3. 2025ARGUS: Hallucination and Omission Evaluation in Video-LLMscited by 16
  4. 2025EventVL: Understand Event Streams via Multimodal Large Language Modelcited by 13
  5. 2026ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understandingcited by 8
  6. 2025VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflectioncited by 6
  7. 2024Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologiescited by 5
  8. 2026Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Groundingcited by 3
  9. 2026Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioningcited by 3
  10. 2024Fine-grained length controllable video captioning with ordinal embeddingscited by 2
  11. 2025Enhancing Surgical Documentation through Multimodal Visual-Temporal Transformers and Generative AIcited by 2
  12. 2026ViSIL: Unified Evaluation of Information Loss in Multimodal Video Captioningcited by 2
5 more
  1. 2025Temporal-Oriented Recipe for Transferring Large Vision-Language Model to Video Understandingcited by 1
  2. 2025Visual Lifelog Retrieval through Captioning-Enhanced Interpretationcited by 1
  3. 2026LFS: Learnable Frame Selector for Event-Aware and Temporally Diverse Video Captioningcited by 1
  4. 2025Toward Scalable Video Narration: A Training-free Approach Using Multimodal Large Language Modelscited by 0
  5. 2026SAVA-X: Ego-to-Exo Imitation Error Detection via Scene-Adaptive View Alignment and Bidirectional Cross View Fusioncited by 0