# Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning

arXiv: https://arxiv.org/abs/2412.12791 · HTML: https://mlsanity.com/paper/2412.12791/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning](https://mlsanity.com/paper/2302.14115/) (2023)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [End-to-End Dense Video Captioning with Parallel Decoding](https://mlsanity.com/paper/2108.07781/) (2021)
- [Multi-modal Dense Video Captioning](https://mlsanity.com/paper/2003.07758/) (2020)
- [Multimodal Pretraining for Dense Video Captioning](https://mlsanity.com/paper/2011.11760/) (2020)
- [Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning](https://mlsanity.com/paper/1804.00100/) (2018)
- [Jointly Localizing and Describing Events for Dense Video Captioning](https://mlsanity.com/paper/1804.08274/) (2018)
- [Weakly Supervised Dense Event Captioning in Videos](https://mlsanity.com/paper/1812.03849/) (2018)
- [Streamlined Dense Video Captioning](https://mlsanity.com/paper/1904.03870/) (2019)
- [End-to-end Dense Video Captioning as Sequence Generation](https://mlsanity.com/paper/2204.08121/) (2022)
- [Gaussian Mixture Proposals with Pull-Push Learning Scheme to Capture Diverse Events for Weakly Supervised Temporal Video Grounding](https://mlsanity.com/paper/2312.16388/) (2023)
- [Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching](https://mlsanity.com/paper/2105.08252/) (2021)
- [Activitynet 2019 Task 3: Exploring Contexts for Dense Captioning Events in Videos](https://mlsanity.com/paper/1907.05092/) (2019)

## Built on by (most cited first, 8 of 8)

- [MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection](https://mlsanity.com/paper/2510.21449/) (2025)
- [Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization](https://mlsanity.com/paper/2506.15980/) (2025)
- [Sali4Vid: Saliency-Aware Video Reweighting and Adaptive Caption Retrieval for Dense Video Captioning](https://mlsanity.com/paper/2509.04602/) (2025)
- [MUVR: A Multi-Modal Untrimmed Video Retrieval Benchmark with Multi-Level Visual Correspondence](https://mlsanity.com/paper/2510.21406/) (2025)
- [LFS: Learnable Frame Selector for Event-Aware and Temporally Diverse Video Captioning](https://mlsanity.com/paper/2601.14594/) (2026)
- [GAT-NeRF: Geometry-Aware-Transformer Enhanced Neural Radiance Fields for High-Fidelity 4D Facial Avatars](https://mlsanity.com/paper/2601.14875/) (2026)
- [SAIL: Similarity-Aware Guidance and Inter-Caption Augmentation-based Learning for Weakly-Supervised Dense Video Captioning](https://mlsanity.com/paper/2603.05437/) (2026)
- [Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning](https://mlsanity.com/paper/2609.04183/) (2026)

## Abstract (arXiv)

Weakly-Supervised Dense Video Captioning (WSDVC) aims to localize and describe all events of interest in a video without requiring annotations of event boundaries. This setting poses a great challenge in accurately locating the temporal location of event, as the relevant supervision is unavailable. Existing methods rely on explicit alignment constraints between event locations and captions, which involve complex event proposal procedures during both training and inference. To tackle this problem, we propose a novel implicit location-caption alignment paradigm by complementary masking, which simplifies the complex event proposal and localization process while maintaining effectiveness. Specifically, our model comprises two components: a dual-mode video captioning module and a mask generation module. The dual-mode video captioning module captures global event information and generates descriptive captions, while the mask generation module generates differentiable positive and negative masks for localizing the events. These masks enable the implicit alignment of event locations and captions by ensuring that captions generated from positively and negatively masked videos are complementary, thereby forming a complete video description. In this way, even under weak supervision, the event location and event caption can be aligned implicitly. Extensive experiments on the public datasets demonstrate that our method outperforms existing weakly-supervised methods and achieves competitive results compared to fully-supervised methods.