# Causal-Story: Local Causal Attention Utilizing Parameter-Efficient Tuning For Visual Story Synthesis

arXiv: https://arxiv.org/abs/2309.09553 · HTML: https://mlsanity.com/paper/2309.09553/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [StoryGAN: A Sequential Conditional GAN for Story Visualization](https://mlsanity.com/paper/1812.02784/) (2018)
- [StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation](https://mlsanity.com/paper/2209.06192/) (2022)
- [Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models](https://mlsanity.com/paper/2211.10950/) (2022)
- [Imagine This! Scripts to Compositions to Videos](https://mlsanity.com/paper/1804.03608/) (2018)
- [Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization](https://mlsanity.com/paper/2110.10834/) (2021)
- [Improving Generation and Evaluation of Visual Stories via Semantic Consistency](https://mlsanity.com/paper/2105.10026/) (2021)
- [Character-Centric Story Visualization via Visual Planning and Token Alignment](https://mlsanity.com/paper/2210.08465/) (2022)
- [Word-Level Fine-Grained Story Visualization](https://mlsanity.com/paper/2208.02341/) (2022)

## Built on by (most cited first, 9 of 9)

- [StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completion](https://mlsanity.com/paper/2404.05979/) (2024)
- [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024)
- [CogCartoon: Towards Practical Story Visualization](https://mlsanity.com/paper/2312.10718/) (2023)
- [ContextualStory: Consistent Visual Storytelling with Spatially-Enhanced and Storyline Context](https://mlsanity.com/paper/2407.09774/) (2024)
- [ViSTA: Visual Storytelling using Multi-modal Adapters for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2506.12198/) (2025)
- [Masked Generative Story Transformer with Character Guidance and Caption Augmentation](https://mlsanity.com/paper/2403.08502/) (2024)
- [Diff2DGS: Reliable Reconstruction of Occluded Surgical Scenes via 2D Gaussian Splatting](https://mlsanity.com/paper/2602.18314/) (2026)
- [FreeStory: Training-Free Character Consistency for Free-Form Visual Storytelling](https://mlsanity.com/paper/2606.25079/) (2026)
- [Sidecar: Training-Free Semantic Reuse for Character-Consistent Free-form Visual Storytelling](https://mlsanity.com/paper/2608.27280/) (2026)

## Abstract (arXiv)

The excellent text-to-image synthesis capability of diffusion models has driven progress in synthesizing coherent visual stories. The current state-of-the-art method combines the features of historical captions, historical frames, and the current captions as conditions for generating the current frame. However, this method treats each historical frame and caption as the same contribution. It connects them in order with equal weights, ignoring that not all historical conditions are associated with the generation of the current frame. To address this issue, we propose Causal-Story. This model incorporates a local causal attention mechanism that considers the causal relationship between previous captions, frames, and current captions. By assigning weights based on this relationship, Causal-Story generates the current frame, thereby improving the global consistency of story generation. We evaluated our model on the PororoSV and FlintstonesSV datasets and obtained state-of-the-art FID scores, and the generated frames also demonstrate better storytelling in visuals.