mlsanityData to 14 Oct 2026

Sali4Vid: Saliency-Aware Video Reweighting and Adaptive Caption Retrieval for Dense Video Captioning

arXiv 2509.04602 ↗First posted 4 Sep 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Dense video captioning aims to temporally localize events in video and generate captions for each event. While recent works propose end-to-end models, they suffer from two limitations: (1) applying timestamp supervision only to text while treating all video frames equally, and (2) retrieving captions from fixed-size video chunks, overlooking scene transitions. To address these, we propose Sali4Vid, a simple yet effective saliency-aware framework. We introduce Saliency-aware Video Reweighting, which converts timestamp annotations into sigmoid-based frame importance weights, and Semantic-based Adaptive Caption Retrieval, which segments videos by frame similarity to capture scene transitions and improve caption retrieval. Sali4Vid achieves state-of-the-art results on YouCook2 and ViTT, demonstrating the benefit of jointly improving video weighting and retrieval for dense video captioning

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

29 of 38 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  7. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  8. 2017Dense-Captioning Events in Videoscited by 793
  9. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  10. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  11. 2018End-to-End Dense Video Captioning with Masked Transformercited by 214
  12. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
17 more
  1. 2020Multi-modal Dense Video Captioningcited by 113
  2. 2018Reconstruction Network for Video Captioningcited by 98
  3. 2020Multimodal Pretraining for Dense Video Captioningcited by 79
  4. 2020A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformercited by 76
  5. 2018Jointly Localizing and Describing Events for Dense Video Captioningcited by 60
  6. 2018Weakly Supervised Dense Event Captioning in Videoscited by 58
  7. 2023SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentariescited by 57
  8. 2019ViSiL: Fine-grained Spatio-Temporal Video Similarity Learningcited by 42
  9. 2024Preserving Multi-Modal Capabilities of Pre-trained VLMs for Improving Vision-Linguistic Compositionalitycited by 17
  10. 2017Video Captioning with Guidance of Multimodal Latent Topicscited by 16
  11. 2024DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinementcited by 13
  12. 2020Dense-Captioning Events in Videos: SYSU Submission to ActivityNet Challenge 2020cited by 9
  13. 2024Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioningcited by 8
  14. 2025VerbDiff: Text-Only Diffusion Models with Enhanced Interaction Awarenesscited by 7
  15. 2023Semi-Supervised Image Captioning by Adversarially Propagating Labeled Datacited by 4
  16. 2025SIDA: Synthetic Image Driven Zero-shot Domain Adaptationcited by 4
  17. 2025SynC: Synthetic Image Caption Dataset Refinement with One-to-many Mapping for Zero-shot Image Captioningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026SAIL: Similarity-Aware Guidance and Inter-Caption Augmentation-based Learning for Weakly-Supervised Dense Video Captioningcited by 1
  2. 2026Stay in your Lane: Role Specific Queries with Overlap Suppression Loss for Dense Video Captioningcited by 1
  3. 2026Follow the Saliency: Supervised Saliency for Retrieval-augmented Dense Video Captioningcited by 1
  4. 2026Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioningcited by 0