mlsanityData to 7 Oct 2026

It's Just Another Day: Unique Video Captioning by Discriminative Prompting

arXiv 2410.11702 ↗First posted 15 Oct 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Long videos contain many repeating actions, events and shots. These repetitions are frequently given identical captions, which makes it difficult to retrieve the exact desired clip using a text search. In this paper, we formulate the problem of unique captioning: Given multiple clips with the same caption, we generate a new caption for each clip that uniquely identifies it. We propose Captioning by Discriminative Prompting (CDP), which predicts a property that can separate identically captioned clips, and use it to generate unique captions. We introduce two benchmarks for unique captioning, based on egocentric footage and timeloop movies - where repeating actions are common. We demonstrate that captions generated by CDP improve text-to-video R@1 by 15% for egocentric videos and 10% in timeloop movies.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 70 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

43 of 70 references.

References · 43

  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  5. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  6. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  7. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  8. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  9. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  10. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  11. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  12. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
31 more
  1. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  2. 2020Rescaling Egocentric Visioncited by 529
  3. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  4. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  5. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  6. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  7. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  8. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  9. 2022Egocentric Video-Language Pretrainingcited by 231
  10. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  11. 2022Learning Video Representations from Large Language Modelscited by 204
  12. 2015A Dataset for Movie Descriptioncited by 195
  13. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  14. 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 161
  15. 2022Language Models with Image Descriptors are Strong Few-Shot Video-Language Learnerscited by 117
  16. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 115
  17. 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
  18. 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
  19. 2024Video ReCap: Recursive Captioning of Hour-Long Videoscited by 88
  20. 2018Localizing Moments in Video with Temporal Languagecited by 83
  21. 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
  22. 2023AutoAD: Movie Description in Contextcited by 50
  23. 2024AutoAD III: The Prequel -- Back to the Pixelscited by 35
  24. 2023CapDet: Unifying Dense Captioning and Open-World Detection Pretrainingcited by 26
  25. 2020Towards Unique and Informative Captioning of Imagescited by 25
  26. 2023Movie101: A New Movie Understanding Benchmarkcited by 25
  27. 2023SpotEM: Efficient Video Search for Episodic Memorycited by 17
  28. 2024Step Differences in Instructional Videocited by 11
  29. 2024Comparison Visual Instruction Tuningcited by 8
  30. 2024FlexCap: Describe Anything in Images in Controllable Detailcited by 7
  31. 2023Set Prediction Guided by Semantic Concepts for Diverse Video Captioningcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 101
  2. 2025Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generationcited by 13
  3. 2024Progress-Aware Video Frame Captioningcited by 10