mlsanityData to 7 Oct 2026

Narration Generation for Cartoon Videos

arXiv 2101.06803 ↗First posted 17 Jan 2021cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Research on text generation from multimodal inputs has largely focused on static images, and less on video data. In this paper, we propose a new task, narration generation, that is complementing videos with narration texts that are to be interjected in several places. The narrations are part of the video and contribute to the storyline unfolding in it. Moreover, they are context-informed, since they include information appropriate for the timeframe of video they cover, and also, do not need to include every detail shown in input scenes, as a caption would. We collect a new dataset from the animated television series Peppa Pig. Furthermore, we formalize the task of narration generation as including two separate tasks, timing and content generation, and present a set of models on the new task.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 48 references that are in the map. First posted on arXiv 17 Jan 2021.

Builds on

22 of 48 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  7. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  8. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  9. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  10. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  11. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  12. 2015A Dataset for Movie Descriptioncited by 195
10 more
  1. 2016Generating Visual Explanationscited by 156
  2. 2016Can neural machine translation do simultaneous translation?cited by 85
  3. 2016Generating Natural Questions About an Imagecited by 56
  4. 2016Leveraging Video Descriptions to Learn Video Question Answeringcited by 55
  5. 2018VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questionscited by 50
  6. 2017Doubly-Attentive Decoder for Multi-modal Neural Machine Translationcited by 46
  7. 2014Coherent Multi-Sentence Video Description with Variable Level of Detailcited by 45
  8. 2016Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measurescited by 36
  9. 2015Using Descriptive Video Services to Create a Large Data Source for Video Annotation Researchcited by 33
  10. 2017Whodunnit? Crime Drama as a Case for Natural Language Understandingcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Learning English with Peppa Pigcited by 8