# Word-Level Fine-Grained Story Visualization

arXiv: https://arxiv.org/abs/2208.02341 · HTML: https://mlsanity.com/paper/2208.02341/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [Cross-Modal Contrastive Learning for Text-to-Image Generation](https://mlsanity.com/paper/2101.04702/) (2021)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [Controllable Text-to-Image Generation](https://mlsanity.com/paper/1909.07083/) (2019)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [ManiGAN: Text-Guided Image Manipulation](https://mlsanity.com/paper/1912.06203/) (2019)
- [Video Generation From Text](https://mlsanity.com/paper/1710.00421/) (2017)
- [StoryGAN: A Sequential Conditional GAN for Story Visualization](https://mlsanity.com/paper/1812.02784/) (2018)
- [Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language](https://mlsanity.com/paper/1810.11919/) (2018)
- [Semantic Object Accuracy for Generative Text-to-Image Synthesis](https://mlsanity.com/paper/1910.13321/) (2019)
- [Semantic Image Synthesis via Adversarial Learning](https://mlsanity.com/paper/1707.06873/) (2017)
- [To Create What You Tell: Generating Videos from Captions](https://mlsanity.com/paper/1804.08264/) (2018)
- [DeepStory: Video Story QA by Deep Embedded Memory Networks](https://mlsanity.com/paper/1707.00836/) (2017)
- [Imagine This! Scripts to Compositions to Videos](https://mlsanity.com/paper/1804.03608/) (2018)
- [Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization](https://mlsanity.com/paper/2110.10834/) (2021)
- [Improving Generation and Evaluation of Visual Stories via Semantic Consistency](https://mlsanity.com/paper/2105.10026/) (2021)
- [Lightweight Generative Adversarial Networks for Text-Guided Image Manipulation](https://mlsanity.com/paper/2010.12136/) (2020)
- [Image-to-Image Translation with Text Guidance](https://mlsanity.com/paper/2002.05235/) (2020)
- [Knowledge Graph Extraction from Videos](https://mlsanity.com/paper/2007.10040/) (2020)

## Built on by (most cited first, 23 of 23)

- [Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond](https://mlsanity.com/paper/2405.03520/) (2024)
- [Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models](https://mlsanity.com/paper/2306.00973/) (2023)
- [TaleCrafter: Interactive Story Visualization with Multiple Characters](https://mlsanity.com/paper/2305.18247/) (2023)
- [Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models](https://mlsanity.com/paper/2211.10950/) (2022)
- [Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models](https://mlsanity.com/paper/2407.02482/) (2024)
- [AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort](https://mlsanity.com/paper/2311.11243/) (2023)
- [Improved Visual Story Generation with Adaptive Context Modeling](https://mlsanity.com/paper/2305.16811/) (2023)
- [StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completion](https://mlsanity.com/paper/2404.05979/) (2024)
- [AesopAgent: Agent-driven Evolutionary System on Story-to-Video Production](https://mlsanity.com/paper/2403.07952/) (2024)
- [Story-Iter: A Training-free Iterative Paradigm for Long Story Visualization](https://mlsanity.com/paper/2410.06244/) (2024)
- [Memory-Driven Text-to-Image Generation](https://mlsanity.com/paper/2208.07022/) (2022)
- [Causal-Story: Local Causal Attention Utilizing Parameter-Efficient Tuning For Visual Story Synthesis](https://mlsanity.com/paper/2309.09553/) (2023)
- [Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Models](https://mlsanity.com/paper/2405.11852/) (2024)
- [CogCartoon: Towards Practical Story Visualization](https://mlsanity.com/paper/2312.10718/) (2023)
- [ContextualStory: Consistent Visual Storytelling with Spatially-Enhanced and Storyline Context](https://mlsanity.com/paper/2407.09774/) (2024)
- [Make-A-Storyboard: A General Framework for Storyboard with Disentangled and Merged Control](https://mlsanity.com/paper/2312.07549/) (2023)
- [ViStoryBench: Comprehensive Benchmark Suite for Story Visualization](https://mlsanity.com/paper/2505.24862/) (2025)
- [AnimeAgent: Is the Multi-Agent via Image-to-Video models a Good Disney Storytelling Artist?](https://mlsanity.com/paper/2602.20664/) (2026)
- [KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Cultures](https://mlsanity.com/paper/2410.19419/) (2024)
- [FairyGen: Storied Cartoon Video from a Single Child-Drawn Character](https://mlsanity.com/paper/2506.21272/) (2025)
- [Learning to Model Multimodal Semantic Alignment for Story Visualization](https://mlsanity.com/paper/2211.07289/) (2022)
- [EvoTale: Continual Character Customization for Expanding Story Worlds](https://mlsanity.com/paper/2603.16285/) (2026)
- [ELECT: Settling Identity Before Rendering for Scalable Story Visualization](https://mlsanity.com/paper/2610.15555/) (2026)

## Abstract (arXiv)

Story visualization aims to generate a sequence of images to narrate each sentence in a multi-sentence story with a global consistency across dynamic scenes and characters. Current works still struggle with output images' quality and consistency, and rely on additional semantic information or auxiliary captioning networks. To address these challenges, we first introduce a new sentence representation, which incorporates word information from all story sentences to mitigate the inconsistency problem. Then, we propose a new discriminator with fusion features and further extend the spatial attention to improve image quality and story consistency. Extensive experiments on different datasets and human evaluation demonstrate the superior performance of our approach, compared to state-of-the-art methods, neither using segmentation masks nor auxiliary captioning networks.