# Narration Generation for Cartoon Videos

arXiv: https://arxiv.org/abs/2101.06803 · HTML: https://mlsanity.com/paper/2101.06803/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 17 Jan 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [A Dataset for Movie Description](https://mlsanity.com/paper/1501.02530/) (2015)
- [Generating Visual Explanations](https://mlsanity.com/paper/1603.08507/) (2016)
- [Can neural machine translation do simultaneous translation?](https://mlsanity.com/paper/1606.02012/) (2016)
- [Generating Natural Questions About an Image](https://mlsanity.com/paper/1603.06059/) (2016)
- [Leveraging Video Descriptions to Learn Video Question Answering](https://mlsanity.com/paper/1611.04021/) (2016)
- [VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions](https://mlsanity.com/paper/1803.07464/) (2018)
- [Doubly-Attentive Decoder for Multi-modal Neural Machine Translation](https://mlsanity.com/paper/1702.01287/) (2017)
- [Coherent Multi-Sentence Video Description with Variable Level of Detail](https://mlsanity.com/paper/1403.6173/) (2014)
- [Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures](https://mlsanity.com/paper/1601.03896/) (2016)
- [Using Descriptive Video Services to Create a Large Data Source for Video Annotation Research](https://mlsanity.com/paper/1503.01070/) (2015)
- [Whodunnit? Crime Drama as a Case for Natural Language Understanding](https://mlsanity.com/paper/1710.11601/) (2017)

## Built on by (most cited first, 1 of 1)

- [Learning English with Peppa Pig](https://mlsanity.com/paper/2202.12917/) (2022)

## Abstract (arXiv)

Research on text generation from multimodal inputs has largely focused on static images, and less on video data. In this paper, we propose a new task, narration generation, that is complementing videos with narration texts that are to be interjected in several places. The narrations are part of the video and contribute to the storyline unfolding in it. Moreover, they are context-informed, since they include information appropriate for the timeframe of video they cover, and also, do not need to include every detail shown in input scenes, as a caption would. We collect a new dataset from the animated television series Peppa Pig. Furthermore, we formalize the task of narration generation as including two separate tasks, timing and content generation, and present a set of models on the new task.