# AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort

arXiv: https://arxiv.org/abs/2311.11243 · HTML: https://mlsanity.com/paper/2311.11243/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 58 references that are in the map.
- First posted on arXiv 19 Nov 2023.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023)
- [Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models](https://mlsanity.com/paper/2305.18292/) (2023)
- [LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models](https://mlsanity.com/paper/2305.13655/) (2023)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [Pixel Difference Networks for Efficient Edge Detection](https://mlsanity.com/paper/2108.07009/) (2021)

## Built on by (most cited first, 30 of 38)

- [Automated Movie Generation via Multi-Agent CoT Planning](https://mlsanity.com/paper/2503.07314/) (2025)
- [LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models](https://mlsanity.com/paper/2403.11627/) (2024)
- [HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives](https://mlsanity.com/paper/2510.20822/) (2025)
- [DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks](https://mlsanity.com/paper/2502.17157/) (2025)
- [MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation](https://mlsanity.com/paper/2411.15262/) (2024)
- [StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration](https://mlsanity.com/paper/2411.04925/) (2024)
- [AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generation](https://mlsanity.com/paper/2406.01388/) (2024)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [TheaterGen: Character Management with LLM for Consistent Multi-turn Image Generation](https://mlsanity.com/paper/2404.18919/) (2024)
- [DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation](https://mlsanity.com/paper/2412.07589/) (2024)
- [Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Rendering](https://mlsanity.com/paper/2508.14461/) (2025)
- [Unbounded: A Generative Infinite Game of Character Life Simulation](https://mlsanity.com/paper/2410.18975/) (2024)
- [AesopAgent: Agent-driven Evolutionary System on Story-to-Video Production](https://mlsanity.com/paper/2403.07952/) (2024)
- [MM-Diff: High-Fidelity Image Personalization via Multi-Modal Condition Integration](https://mlsanity.com/paper/2403.15059/) (2024)
- [CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives](https://mlsanity.com/paper/2605.12496/) (2026)
- [Text-to-Image Synthesis: A Decade Survey](https://mlsanity.com/paper/2411.16164/) (2024)
- [CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion](https://mlsanity.com/paper/2408.17424/) (2024)
- [AnimeGamer: Infinite Anime Life Simulation with Next Game State Prediction](https://mlsanity.com/paper/2504.01014/) (2025)
- [iMontage: Unified, Versatile, Highly Dynamic Many-to-many Image Generation](https://mlsanity.com/paper/2511.20635/) (2025)
- [ViStoryBench: Comprehensive Benchmark Suite for Story Visualization](https://mlsanity.com/paper/2505.24862/) (2025)
- [Training-free Color-Style Disentanglement for Constrained Text-to-Image Synthesis](https://mlsanity.com/paper/2409.02429/) (2024)
- [KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Cultures](https://mlsanity.com/paper/2410.19419/) (2024)
- [VisAgent: Narrative-Preserving Story Visualization Framework](https://mlsanity.com/paper/2503.02399/) (2025)
- [PAVE: Patching and Adapting Video Large Language Models](https://mlsanity.com/paper/2503.19794/) (2025)
- [CineVerse: Consistent Keyframe Synthesis for Cinematic Scene Composition](https://mlsanity.com/paper/2504.19894/) (2025)
- [FairyGen: Storied Cartoon Video from a Single Child-Drawn Character](https://mlsanity.com/paper/2506.21272/) (2025)
- [LogiStory: A Logic-Aware Framework for Multi-Image Story Visualization](https://mlsanity.com/paper/2603.28082/) (2026)
- [CoCoIns: Consistent Subject Generation via Contrastive Instantiated Concepts](https://mlsanity.com/paper/2503.24387/) (2025)
- [FlexMUSE: Multimodal Unification and Semantics Enhancement Framework with Flexible interaction for Creative Writing](https://mlsanity.com/paper/2508.16230/) (2025)
- [CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Models](https://mlsanity.com/paper/2508.20640/) (2025)

## Abstract (arXiv)

Story visualization aims to generate a series of images that match the story described in texts, and it requires the generated images to satisfy high quality, alignment with the text description, and consistency in character identities. Given the complexity of story visualization, existing methods drastically simplify the problem by considering only a few specific characters and scenarios, or requiring the users to provide per-image control conditions such as sketches. However, these simplifications render these methods incompetent for real applications. To this end, we propose an automated story visualization system that can effectively generate diverse, high-quality, and consistent sets of story images, with minimal human interactions. Specifically, we utilize the comprehension and planning capabilities of large language models for layout planning, and then leverage large-scale text-to-image models to generate sophisticated story images based on the layout. We empirically find that sparse control conditions, such as bounding boxes, are suitable for layout planning, while dense control conditions, e.g., sketches and keypoints, are suitable for generating high-quality image content. To obtain the best of both worlds, we devise a dense condition generation module to transform simple bounding box layouts into sketch or keypoint control conditions for final image generation, which not only improves the image quality but also allows easy and intuitive user interactions. In addition, we propose a simple yet effective method to generate multi-view consistent character images, eliminating the reliance on human labor to collect or draw character images.