mlsanityData to 7 Oct 2026

Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models

arXiv 2211.10950 ↗First posted 20 Nov 2022cs.CVcited by 69

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Conditioned diffusion models have demonstrated state-of-the-art text-to-image synthesis capacity. Recently, most works focus on synthesizing independent images; While for real-world applications, it is common and necessary to generate a series of coherent images for story-stelling. In this work, we mainly focus on story visualization and continuation tasks and propose AR-LDM, a latent diffusion model auto-regressively conditioned on history captions and generated images. Moreover, AR-LDM can generalize to new characters through adaptation. To our best knowledge, this is the first work successfully leveraging diffusion models for coherent visual story synthesizing. Quantitative results show that AR-LDM achieves SoTA FID scores on PororoSV, FlintstonesSV, and the newly introduced challenging dataset VIST containing natural images. Large-scale human evaluations show that AR-LDM has superior performance in terms of quality, relevance, and consistency.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 37 of its 54 references that are in the map. First posted on arXiv 20 Nov 2022.

Builds on

37 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2022Generative Adversarial Networkscited by 5,603
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
22 more
  1. 2014Going Deeper with Convolutionscited by 4,231
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  4. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  6. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  7. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  8. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  9. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  10. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  11. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  12. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  13. 2016Visual Storytellingcited by 176
  14. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  15. 2018StoryGAN: A Sequential Conditional GAN for Story Visualizationcited by 137
  16. 2022StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuationcited by 74
  17. 2017DeepStory: Video Story QA by Deep Embedded Memory Networkscited by 67
  18. 2018Imagine This! Scripts to Compositions to Videoscited by 54
  19. 2021Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualizationcited by 51
  20. 2021Improving Generation and Evaluation of Visual Stories via Semantic Consistencycited by 43
  21. 2022Character-Centric Story Visualization via Visual Planning and Token Alignmentcited by 32
  22. 2022Word-Level Fine-Grained Story Visualizationcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 27; 2025: 18; 2026: 7.

69 papers.

Sort:

By area:

  1. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  2. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  3. 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 112
  4. 2023HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generationcited by 104
  5. 2025One-Minute Video Generation with Test-Time Trainingcited by 100
  6. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 85
  7. 2024MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizercited by 80
  8. 2023Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Modelscited by 76
  9. 2023TaleCrafter: Interactive Story Visualization with Multiple Characterscited by 71
  10. 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 70
  11. 2023Directed Diffusion: Direct Control of Object Placement through Attention Guidancecited by 69
  12. 2023Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllabilitycited by 66
38 more
  1. 2024Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Modelscited by 54
  2. 2024DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusioncited by 46
  3. 2025Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Modelcited by 46
  4. 2024Mind the Time: Temporally-Controlled Multi-Event Video Generationcited by 39
  5. 2025VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual interventioncited by 33
  6. 2024StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaborationcited by 28
  7. 2023Zero-shot Generation of Coherent Storybook from Plain Text Story using Diffusion Modelscited by 26
  8. 2023StoryGPT-V: Large Language Models as Consistent Story Visualizerscited by 25
  9. 2024MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuningcited by 25
  10. 2023Improved Visual Story Generation with Adaptive Context Modelingcited by 22
  11. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  12. 2023Synthesizing Long-Term Human Motions with Diffusion Models via Coherent Samplingcited by 19
  13. 2024StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completioncited by 19
  14. 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Drivingcited by 18
  15. 2024Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generationcited by 17
  16. 2025Generative AI for Cel-Animation: A Surveycited by 17
  17. 2023OpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluationcited by 16
  18. 2024DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generationcited by 16
  19. 2024Story-Iter: A Training-free Iterative Paradigm for Long Story Visualizationcited by 12
  20. 2024Pixel is a Barrier: Diffusion Models Are More Adversarially Robust Than We Thinkcited by 10
  21. 2023Causal-Story: Local Causal Attention Utilizing Parameter-Efficient Tuning For Visual Story Synthesiscited by 9
  22. 2023Image Content Generation with Causal Reasoningcited by 9
  23. 2024Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Modelscited by 9
  24. 2023CogCartoon: Towards Practical Story Visualizationcited by 8
  25. 2024ContextualStory: Consistent Visual Storytelling with Spatially-Enhanced and Storyline Contextcited by 8
  26. 2024One missing piece in Vision and Language: A Survey on Comics Understandingcited by 8
  27. 2023Make-A-Storyboard: A General Framework for Storyboard with Disentangled and Merged Controlcited by 7
  28. 2024CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusioncited by 7
  29. 2025Object Isolated Attention for Consistent Story Visualizationcited by 7
  30. 2025SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistencycited by 7
  31. 2024Many-to-many Image Generation with Auto-regressive Diffusion Modelscited by 6
  32. 2024Generating Coherent Sequences of Visual Illustrations for Real-World Manual Taskscited by 6
  33. 2024HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generationcited by 5
  34. 2024RefDrop: Controllable Consistency in Image or Video Generation via Reference Feature Guidancecited by 4
  35. 2025VisAgent: Narrative-Preserving Story Visualization Frameworkcited by 4
  36. 2025VinaBench: Benchmark for Faithful and Consistent Visual Narrativescited by 4
  37. 2024Contrastive Sequential-Diffusion Learning: Non-linear and Multi-Scene Instructional Video Synthesiscited by 3
  38. 2024Improving Multi-Subject Consistency in Open-Domain Image Generation with Isolation and Reposition Attentioncited by 3

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Audit & Repair: An Agentic Framework for Consistent Story Visualization in Text-to-Image Diffusion Models
  2. 2023SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models
  3. 2024ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction
  4. 2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
  5. 2024Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling
  6. 2023Text-driven Visual Synthesis with Latent Diffusion Prior
  7. 2023Progressive Text-to-Image Diffusion with Soft Latent Direction
  8. 2026RealDiffusion: Physics-informed Attention for Multi-character Storybook Generation