# Grounded Objects and Interactions for Video Captioning

arXiv: https://arxiv.org/abs/1711.06354 · HTML: https://mlsanity.com/paper/1711.06354/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2017.

## Built on by (most cited first, 2 of 2)

- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [Decoupled Spatial Temporal Graphs for Generic Visual Grounding](https://mlsanity.com/paper/2103.10191/) (2021)

## Abstract (arXiv)

We address the problem of video captioning by grounding language generation on object interactions in the video. Existing work mostly focuses on overall scene understanding with often limited or no emphasis on object interactions to address the problem of video understanding. In this paper, we propose SINet-Caption that learns to generate captions grounded over higher-order interactions between arbitrary groups of objects for fine-grained video understanding. We discuss the challenges and benefits of such an approach. We further demonstrate state-of-the-art results on the ActivityNet Captions dataset using our model, SINet-Caption based on this approach.