# Dense Video Captioning using Graph-based Sentence Summarization

arXiv: https://arxiv.org/abs/2506.20583 · HTML: https://mlsanity.com/paper/2506.20583/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 60 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018)
- [Exploring Visual Relationship for Image Captioning](https://mlsanity.com/paper/1809.07041/) (2018)
- [End-to-End Dense Video Captioning with Masked Transformer](https://mlsanity.com/paper/1804.00819/) (2018)
- [Sequence to Sequence -- Video to Text](https://mlsanity.com/paper/1505.00487/) (2015)
- [Reconstruction Network for Video Captioning](https://mlsanity.com/paper/1803.11438/) (2018)
- [Translating Videos to Natural Language Using Deep Recurrent Neural Networks](https://mlsanity.com/paper/1412.4729/) (2014)
- [Describing Videos by Exploiting Temporal Structure](https://mlsanity.com/paper/1502.08029/) (2015)
- [Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning](https://mlsanity.com/paper/1804.00100/) (2018)
- [Jointly Localizing and Describing Events for Dense Video Captioning](https://mlsanity.com/paper/1804.08274/) (2018)
- [Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks](https://mlsanity.com/paper/1510.07712/) (2015)
- [Jointly Modeling Embedding and Translation to Bridge Video and Language](https://mlsanity.com/paper/1505.01861/) (2015)
- [Less Is More: Picking Informative Frames for Video Captioning](https://mlsanity.com/paper/1803.01457/) (2018)
- [Semantic Compositional Networks for Visual Captioning](https://mlsanity.com/paper/1611.08002/) (2016)
- [Video Captioning via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1711.11135/) (2017)
- [Spatiotemporal Co-attention Recurrent Neural Networks for Human-Skeleton Motion Prediction](https://mlsanity.com/paper/1909.13245/) (2019)
- [Video Captioning with Transferred Semantic Attributes](https://mlsanity.com/paper/1611.07675/) (2016)
- [Differential Recurrent Neural Networks for Action Recognition](https://mlsanity.com/paper/1504.06678/) (2015)
- [Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition](https://mlsanity.com/paper/1811.00270/) (2018)
- [Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning](https://mlsanity.com/paper/1511.03476/) (2015)
- [Weakly Supervised Dense Video Captioning](https://mlsanity.com/paper/1704.01502/) (2017)

## Built on by (most cited first, 1 of 1)

- [ProCap: Prominence-guided Object Rectification for Faithful and Comprehensive Video Captioning](https://mlsanity.com/paper/2607.21022/) (2026)

## Abstract (arXiv)

Recently, dense video captioning has made attractive progress in detecting and captioning all events in a long untrimmed video. Despite promising results were achieved, most existing methods do not sufficiently explore the scene evolution within an event temporal proposal for captioning, and therefore perform less satisfactorily when the scenes and objects change over a relatively long proposal. To address this problem, we propose a graph-based partition-and-summarization (GPaS) framework for dense video captioning within two stages. For the ``partition" stage, a whole event proposal is split into short video segments for captioning at a finer level. For the ``summarization" stage, the generated sentences carrying rich description information for each segment are summarized into one sentence to describe the whole event. We particularly focus on the ``summarization" stage, and propose a framework that effectively exploits the relationship between semantic words for summarization. We achieve this goal by treating semantic words as nodes in a graph and learning their interactions by coupling Graph Convolutional Network (GCN) and Long Short Term Memory (LSTM), with the aid of visual cues. Two schemes of GCN-LSTM Interaction (GLI) modules are proposed for seamless integration of GCN and LSTM. The effectiveness of our approach is demonstrated via an extensive comparison with the state-of-the-arts methods on the two benchmarks ActivityNet Captions dataset and YouCook II dataset.