# Video In Sentences Out

arXiv: https://arxiv.org/abs/1408.6418 · HTML: https://mlsanity.com/paper/1408.6418/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2014.

## Built on by (most cited first, 7 of 7)

- [Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Vision](https://mlsanity.com/paper/2103.04037/) (2021)
- [Trajectory Prediction with Linguistic Representations](https://mlsanity.com/paper/2110.09741/) (2021)
- [A Survey on Integrated Sensing, Communication, and Computation](https://mlsanity.com/paper/2408.08074/) (2024)
- [A Comprehensive Review of the Video-to-Text Problem](https://mlsanity.com/paper/2103.14785/) (2021)
- [Dual-Level Decoupled Transformer for Video Captioning](https://mlsanity.com/paper/2205.03039/) (2022)
- [CLearViD: Curriculum Learning for Video Description](https://mlsanity.com/paper/2311.04480/) (2023)
- [Multi-Modal interpretable automatic video captioning](https://mlsanity.com/paper/2411.06872/) (2024)

## Abstract (arXiv)

We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it. Action class is rendered as a verb, participant objects as noun phrases, properties of those objects as adjectival modifiers in those noun phrases, spatial relations between those participants as prepositional phrases, and characteristics of the event as prepositional-phrase adjuncts and adverbial modifiers. Extracting the information needed to render these linguistic entities requires an approach to event recognition that recovers object tracks, the trackto-role assignments, and changing body posture.