# Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning

arXiv: https://arxiv.org/abs/1804.00100 · HTML: https://mlsanity.com/paper/1804.00100/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2018.

## Built on by (most cited first, 30 of 68)

- [Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning](https://mlsanity.com/paper/2302.14115/) (2023)
- [VTimeLLM: Empower LLM to Grasp Video Moments](https://mlsanity.com/paper/2311.18445/) (2023)
- [End-to-End Dense Video Captioning with Parallel Decoding](https://mlsanity.com/paper/2108.07781/) (2021)
- [End-to-end Generative Pretraining for Multimodal Video Captioning](https://mlsanity.com/paper/2201.08264/) (2022)
- [Streaming Dense Video Captioning](https://mlsanity.com/paper/2404.01297/) (2024)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [AutoAD: Movie Description in Context](https://mlsanity.com/paper/2303.16899/) (2023)
- [AutoAD II: The Sequel -- Who, When, and What in Movie Audio Description](https://mlsanity.com/paper/2310.06838/) (2023)
- [MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenes](https://mlsanity.com/paper/2203.05203/) (2022)
- [UnIVAL: Unified Model for Image, Video, Audio and Language Tasks](https://mlsanity.com/paper/2307.16184/) (2023)
- [Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challenges](https://mlsanity.com/paper/2309.13925/) (2023)
- [VideoXum: Cross-modal Visual and Textural Summarization of Videos](https://mlsanity.com/paper/2303.12060/) (2023)
- [Do You Remember? Dense Video Captioning with Cross-Modal Memory Retrieval](https://mlsanity.com/paper/2404.07610/) (2024)
- [AutoAD III: The Prequel -- Back to the Pixels](https://mlsanity.com/paper/2404.14412/) (2024)
- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [End-to-end Dense Video Captioning as Sequence Generation](https://mlsanity.com/paper/2204.08121/) (2022)
- [Towards Diverse Paragraph Captioning for Untrimmed Videos](https://mlsanity.com/paper/2105.14477/) (2021)
- [OmniVid: A Generative Framework for Universal Video Understanding](https://mlsanity.com/paper/2403.17935/) (2024)
- [Fine-grained Audible Video Description](https://mlsanity.com/paper/2303.15616/) (2023)
- [Unifying Event Detection and Captioning as Sequence Generation via Pre-Training](https://mlsanity.com/paper/2207.08625/) (2022)
- [Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocols](https://mlsanity.com/paper/2311.02538/) (2023)
- [Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generation](https://mlsanity.com/paper/2504.01020/) (2025)
- [DistinctAD: Distinctive Audio Description Generation in Contexts](https://mlsanity.com/paper/2411.18180/) (2024)
- [Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos](https://mlsanity.com/paper/2311.16444/) (2023)
- [SlowFocus: Enhancing Fine-grained Temporal Understanding in Video LLM](https://mlsanity.com/paper/2602.03589/) (2026)
- [Exploiting Auxiliary Caption for Video Grounding](https://mlsanity.com/paper/2301.05997/) (2023)
- [A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes](https://mlsanity.com/paper/2403.07469/) (2024)
- [Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting](https://mlsanity.com/paper/2504.05541/) (2025)
- [Controllable Video Captioning with an Exemplar Sentence](https://mlsanity.com/paper/2112.01073/) (2021)
- [Enhancing Traffic Safety with Parallel Dense Video Captioning for End-to-End Event Analysis](https://mlsanity.com/paper/2404.08229/) (2024)

## Abstract (arXiv)

Dense video captioning is a newly emerging task that aims at both localizing and describing all events in a video. We identify and tackle two challenges on this task, namely, (1) how to utilize both past and future contexts for accurate event proposal predictions, and (2) how to construct informative input to the decoder for generating natural event descriptions. First, previous works predominantly generate temporal event proposals in the forward direction, which neglects future video context. We propose a bidirectional proposal method that effectively exploits both past and future contexts to make proposal predictions. Second, different events ending at (nearly) the same time are indistinguishable in the previous works, resulting in the same captions. We solve this problem by representing each event with an attentive fusion of hidden states from the proposal module and video contents (e.g., C3D features). We further propose a novel context gating mechanism to balance the contributions from the current event and its surrounding contexts dynamically. We empirically show that our attentively fused event representation is superior to the proposal hidden states or video contents alone. By coupling proposal and captioning modules into one unified framework, our model outperforms the state-of-the-arts on the ActivityNet Captions dataset with a relative gain of over 100% (Meteor score increases from 4.82 to 9.65).