# StoryGAN: A Sequential Conditional GAN for Story Visualization

arXiv: https://arxiv.org/abs/1812.02784 · HTML: https://mlsanity.com/paper/1812.02784/

## Key facts

- Cited by 138 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2018.

## Built on by (most cited first, 30 of 138)

- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning](https://mlsanity.com/paper/2309.15091/) (2023)
- [MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale](https://mlsanity.com/paper/2412.05237/) (2024)
- [Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond](https://mlsanity.com/paper/2405.03520/) (2024)
- [One-Minute Video Generation with Test-Time Training](https://mlsanity.com/paper/2504.05298/) (2025)
- [FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention](https://mlsanity.com/paper/2407.19918/) (2024)
- [Automated Movie Generation via Multi-Agent CoT Planning](https://mlsanity.com/paper/2503.07314/) (2025)
- [MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer](https://mlsanity.com/paper/2401.10208/) (2024)
- [Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models](https://mlsanity.com/paper/2306.00973/) (2023)
- [Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions](https://mlsanity.com/paper/2308.04152/) (2023)
- [Make-A-Story: Visual Memory Conditioned Consistent Story Generation](https://mlsanity.com/paper/2211.13319/) (2022)
- [StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation](https://mlsanity.com/paper/2209.06192/) (2022)
- [TaleCrafter: Interactive Story Visualization with Multiple Characters](https://mlsanity.com/paper/2305.18247/) (2023)
- [SEED-Story: Multimodal Long Story Generation with Large Language Model](https://mlsanity.com/paper/2407.08683/) (2024)
- [Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models](https://mlsanity.com/paper/2211.10950/) (2022)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model](https://mlsanity.com/paper/2404.09967/) (2024)
- [VideoStudio: Generating Consistent-Content and Multi-Scene Videos](https://mlsanity.com/paper/2401.01256/) (2024)
- [The Chosen One: Consistent Characters in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2311.10093/) (2023)
- [Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models](https://mlsanity.com/paper/2407.02482/) (2024)
- [Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization](https://mlsanity.com/paper/2110.10834/) (2021)
- [DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion](https://mlsanity.com/paper/2407.12899/) (2024)
- [Improving Generation and Evaluation of Visual Stories via Semantic Consistency](https://mlsanity.com/paper/2105.10026/) (2021)
- [SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis](https://mlsanity.com/paper/2304.14573/) (2023)
- [StoryMem: Multi-shot Long Video Storytelling with Memory](https://mlsanity.com/paper/2512.19539/) (2025)
- [AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort](https://mlsanity.com/paper/2311.11243/) (2023)
- [Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation](https://mlsanity.com/paper/2408.09787/) (2024)
- [Character-Centric Story Visualization via Visual Planning and Token Alignment](https://mlsanity.com/paper/2210.08465/) (2022)
- [StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration](https://mlsanity.com/paper/2411.04925/) (2024)
- [StoryBench: A Multifaceted Benchmark for Continuous Story Visualization](https://mlsanity.com/paper/2308.11606/) (2023)

## Abstract (arXiv)

We propose a new task, called Story Visualization. Given a multi-sentence paragraph, the story is visualized by generating a sequence of images, one for each sentence. In contrast to video generation, story visualization focuses less on the continuity in generated images (frames), but more on the global consistency across dynamic scenes and characters -- a challenge that has not been addressed by any single-image or video generation methods. We therefore propose a new story-to-image-sequence generation model, StoryGAN, based on the sequential conditional GAN framework. Our model is unique in that it consists of a deep Context Encoder that dynamically tracks the story flow, and two discriminators at the story and image levels, to enhance the image quality and the consistency of the generated sequences. To evaluate the model, we modified existing datasets to create the CLEVR-SV and Pororo-SV datasets. Empirically, StoryGAN outperforms state-of-the-art models in image quality, contextual consistency metrics, and human evaluation.