# Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences

arXiv: https://arxiv.org/abs/1911.10460 · HTML: https://mlsanity.com/paper/1911.10460/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2019.

## Built on by (most cited first, 11 of 11)

- [Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions](https://mlsanity.com/paper/2111.10337/) (2021)
- [EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone](https://mlsanity.com/paper/2307.05463/) (2023)
- [Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2212.04634/) (2022)
- [ReelFramer: Human-AI Co-Creation for News-to-Video Translation](https://mlsanity.com/paper/2304.09653/) (2023)
- [What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluation](https://mlsanity.com/paper/2408.14622/) (2024)
- [Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval](https://mlsanity.com/paper/2308.04343/) (2023)
- [TeViS:Translating Text Synopses to Video Storyboards](https://mlsanity.com/paper/2301.00135/) (2022)
- [PrevizWhiz: Combining Rough 3D Scenes and 2D Video to Guide Generative Video Previsualization](https://mlsanity.com/paper/2602.03838/) (2026)
- [Automatic Comic Generation with Stylistic Multi-page Layouts and Emotion-driven Text Balloon Generation](https://mlsanity.com/paper/2101.11111/) (2021)
- [Multi-Modal Experience Inspired AI Creation](https://mlsanity.com/paper/2209.02427/) (2022)
- [VScript: Controllable Script Generation with Visual Presentation](https://mlsanity.com/paper/2203.00314/) (2022)

## Abstract (arXiv)

A storyboard is a sequence of images to illustrate a story containing multiple sentences, which has been a key process to create different story products. In this paper, we tackle a new multimedia task of automatic storyboard creation to facilitate this process and inspire human artists. Inspired by the fact that our understanding of languages is based on our past experience, we propose a novel inspire-and-create framework with a story-to-image retriever that selects relevant cinematic images for inspiration and a storyboard creator that further refines and renders images to improve the relevancy and visual consistency. The proposed retriever dynamically employs contextual information in the story with hierarchical attentions and applies dense visual-semantic matching to accurately retrieve and ground images. The creator then employs three rendering steps to increase the flexibility of retrieved images, which include erasing irrelevant regions, unifying styles of images and substituting consistent characters. We carry out extensive experiments on both in-domain and out-of-domain visual story datasets. The proposed model achieves better quantitative performance than the state-of-the-art baselines for storyboard creation. Qualitative visualizations and user studies further verify that our approach can create high-quality storyboards even for stories in the wild.